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ABSTRACT 

This report describes results of a demonstration 
project carried out in four cities during 1971-72* The project aimed 
at exploring the feasibility and impact of two different forms of 
money incentives payments. In one form the ••Tfeacl»rHOnly" model ~ 
the teachers in a school nere of fered a series of bonuses ranging 
from $150 to $600 per class per subject, depending on the amount of 
gain shown by their students on standardized tests of reading and 
niath. In the ••Parent-TeacherS model, the same bonus offer was made to 
teachers: but in addition, there were cash payments of $12*50 to $50 
of fered to each parent, depending on the mean gain shown by all of 
the students in their child's class. Quantitative data were collected 
from students, teachers, and parents at two points xn time. This data 
included the results of questionnaires tapping attitude and behavior 
information, interview results, tallies from systematic observation, 
and standardized achievement test results. The data were analyzed by 
comparing the mean of an EXP school (one %rtiich had been offered 
incentives) to the mean of a matched CON school in the same city. The 
interpretation was restricted by problems inherent in the design and 
time schedule of the project. Although it is possible that results 
reflect factors other than the incentives offer, th6 achievement 
gains observed for students in the Parent-Teacher model were 
substantially greater than those of the control grotp. Differences in 
achievement gain b^ween the Teacher-Gifily model and the control group 
were n^ligible. other results from the analysis of the attitude and 
behavior data also are presented and discussed, but no simple 
patterns were evident in these results. (Author) . 
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Executive Summary 



Introduction 

The Office of Education designed and implemented a one-year project 
to demonstrate the feasibility of offering incentives to parents and' 
teachers or to the teachers only. Separate process arid impact evaluations 
of this demonstration were conducted under contract to the Office of Pro- 
gram Planning and Evaluation in the Office of Education. The process 
evaluation was conducted by Education Turnkey Systems, Inc. and can be 
fqimd in two reports: Final Report, July 1, 1972^ and Final Evaluation 
Report, July 31, 1972. This report presents the results of the impact 
evaluation conducted by the Plahar Corporation. 

Project Description 

two jjOLcentive models were ^nployed in four medium-sized cities for one 
year In this project. In Cincinnatiy Ohio ^ arid Jacksonville, Florida, in- 
centives of lip to $600 per subject were offered to teachers, (Teacher Only 
model), deperiding entirely iipon their classroom's performance on standard- 
ized Reading and Mathematics achievement tests. In Oakland, California, 
and San Antoriio, Texas, 'incentives of up to $600 per subject were of feted 
to teachers and incentives of up to $50 per subject were offered to parents, 
(Parent Teacher model) depending eritirely upon the classroom's perfbrmarice 
on standardized Reading and Mathdnatics achievement tests. For each class- 
room a predicted growth rate was calculated in both subjects, and the 
amount of payinent to the parents and teachers depended tipon the degree to 
which the classroom exceeded: the predicted growth rate. 

In each city there was one experimental school in which the incentives 
were offered and a matched control school. All schools in the project 
were elementary schools. Grades 1-6, with from 500 to 700 students, most 
of whom were low -income and underachiev The project began in September, 

1971 and ended in May, 1972. 
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Impact Evaluation Objectives ^ 

This project had extremely limited objectives: Only two out of a 
multitude of incentive models were implemented; only four cities with; one 
building per city participated in the implementation; and only the feasi- 
bility of incentives, was to be evaluated. Because so little is known 
about what happens when incentives are of f ered to parents and teachers in 
real world contexts^ this evaluation covered a wide range of jparticipants 
and outcomes, in both the experimental arid coritrol schools. 

The major questions guiding the evaluatiori were: 

'-What are the positive and negative results on student attitudes, 
behavior arid achievement and parent and: teacher attitudes and be- 
havior of offering incentives in education, and what are the im- 
plications of these results for future Federal research?" 

Findings 

The results that are presented here must be accepted very cautiously by 
the reader. The project was delayed substantially by contractual problems, 
so that the teachers and parents did not sign their contracts until the 
school year was half over. Also, administrative delays rprevented a full 
assessment of student, teacher and parent attitudes i 

Student Achievement Results In Reading, there was an educationally 
significant increase orily in Oakland, In Cincirinati, thervr was an apparent 
decrease in learning rate. 

In Mathematics, there were increases in learning in Oakland arid San 
Antonio. 

Student A ttitudes Results. In Oakland there was a slight positive im- 
pact upon studerit attitudes. In Sjan Antonio, there was ari equally slight 
negative im- acti 

Student Behavior Results. No observed differences at any site. 

Teacher A ttitude Results . In Cincinnati, there was a slight negative 
impact. In Jacksonville, there was a slight positive impact. In San 
Aritonio, there was a slight negative impact in which the teachers favored 
more traditional teachirig techniques. 
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Teacher Behavior Results > In Cincinnati arid in Jacksonville there is 
a trend toward inbre traditional classroom behaviors. In Oakland arid San 
Antonio there is a slight trend toward traditionalism. 

Parent Attitude Results, In Cincinnati and Jacksonville there is a 
negative trends In Oakland and San Antonio, there is a positive trend 

^ Methodologicai Findings , 

Student Achievement, the data were analyzed for the fan-spread 
phenomenon and the results of the analysis did not change the above findings 
However, the achievement results are probably influenced by regression arti- 
facts of an unknown magnitude. 

Project Design, The small number of units and the nohraridpm 
assignment of schools to treatment conditions limit the generalizatiohs 
that can be drawn from this project i 

Ins t rument at ion i There was not enough time for the development, 
pretesting and clearance of the forms used in assessing attitudes i 

Implementation, Unresolved questions of the funding authority 
hindered the full implementation of the ihcentiva models and the evaluation. 

ConclusionSr 

Single iSite Cbriclusidns > 

p In Cincinnati, the offer of incentives to teachers only^^ 
negative impiact, _ 

6 In Jacksonville, the offer incentives to teachers only had a 
mixed impact, both positive arid negative, 

o Iri Oaklarid,. the offer of ihcehtives to parents and teachers had- 
a positive impact, 

o In San Antonio, the offer of incentives to parents arid teachers 
had a slight positive impact. 
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Incentive Model Cohclusldns, 

o ' The offer of Incentives to teachers only had an overall negative 
Impact • 

o The offer of incentives to parents and teachers had ah overall 
slightly positive impact • 

Logistics and Management Conclusions. / 

" - / 

o Single payment formulas, e.g.^ one yearns growth, are going to 
turn out to be unfair in most cases. 

p School district data files are imreilable and are not a trust- 
worthy source of primary or secondary evaluation material; 

6 Research projects become a part of the power conflicts and con- 
fusions in a school district.. 

o The traditional communicatipns system between hbihes and schools 
in a district caimot bear the burdens imposed upon It by innova- 
tive projects. 

o QE is not administratively structured to conduct adequately field 
projects of this type. 

Recommendations 

1. The teacher Only incentives model should not be further researched: 
or advocated; 

2. The Parent Teacher model should be further researched. 

3* Field research of incentives in educatiph should provide for 
changes in the distribution of authority in school buildings 
and districts. 

4. Field projects should have three phases, with each phase lasting 
at least one year. 

5. As a means of gaining their responsible conmiitment to a field 
project initiated at the Federal level, school districts should 
share in funding the project. 
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6* Federal agencies that undertake field projects that are quasi- 
experimental in design and that necessitate a thorough evalua- 
tion must be intemaiiy structured so that the project officer 
has extensive authority,. Fiscal, site selection, operatiohai 
and evaluation decisions in projects of this type must be made 
swiftly and authoritatively. 
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CHAPTER I 
PROJECT DESCRIPTION 



Background 

Traditional educational methods do not seem to be effective for the low- 
income, underachieving student* Entering the public school system from a home 
of economic deprivation, this child is in a position of disadvantage from the 
very beginning. As he progresses through elementary and secondary school, the 
gap between the student of low-income background and other students tends to 
widen. The disparity is manifested primarily in terms of academic achieve- 
ment, but there are major secondary effects in the areas of social adjust-- 
ment, self-concept, career expectations, and level of aspiration. The child 
from a poverty background starts lower, learns less rapidly, and pr^Hts least 
educationally and economically from the traditional educational program 
offered by the public schools. 

In personal, psychic terms the toll on the student is appalling. There 
is a Kafka-like atmosphere for the child. He plays a game the rules of 
which he does hot understand, the purpose of which he is never able to compre- 
hend, and the outcomes of which are almost certainly going to be unfavorable 
for him. It is small wonder that the low-income student soon withdraws, 
first spiritually, and then, when he can, bodily as well. 

Social conscience and the simple economics of conserving human resources 
demand that methods be found to help the low-income underachieves to break 
out of this grim cycle and to perform academically on a par with his more 
affluent and fortunate contemporaries. Numerous educational innovations have 
been tried. At the Federal level, these innovations are relatively new; but 
they are increasing. Head Start and Home Start, aimed at lessening the 
initial disadvantage at a relatively early age represent one approach. 
Experimentation with new instructional techniques and media, as in the OEO 
performance contracting field experiment, is another. A third, which is the 
subject of the present study, involves the use of incentives with no control 
over program content. The basic assumption is that if incentives are offered 



to teachers and parents, their motivation will increase and hence, the achieve- 
ment of economically deprived students will increase • 

Intrinsic rewards, mastery of a discipline or task, eto.^ have tradi- 
tionally been advocated by educators as the proper route toward more effec- 
tive learning. However, this intrinsic reward system, upon closer examina- 
tion, has been supported, indeed sustained, by an intricate system of 
rewards that are extrinsic to the act of learning. Extrinsic incentives are 
not new in the public schools. The educational system abounds with induce- 
ments and rewards to the students. There are "stars", grades, certificates, 
and other such tokens which are extrinsically valuable and may be exchanged 
for teacher and parent approval, peer esteem, or other psychic intangibles. 
Ultimately, academic success is tied to such long-term extrinsic incentives 
as a better job, wider opportunities for self-fulfillment, and a richer 
quality of life. This tacit assumption that education is directly related to 
upward mobility is shared by some teachers, parents, and the community as a 
whole. There is an ingrained tenet in the public school system that academic 
success leads, or should lead, to material reward. 

For low-income children, however, the link between present achievement 
and future payoff is at best tenuous. Consequently, much of the recent 
experimentation with incentives in education has sought to use rewards syste- 
matically and to develop ways for making the reward structure, explicit and 
more powerfully operative in the teaching-learning situation. Frequently 
this has involved supplying specific, immediate, and tangible rewards for 
classroom achievement. In a majority of the previous cases, the student has 
been the recipient; in the case of this project, the performance incentives 
were offered to teachers or to parents and teachers. 

During the school year 1970-71, performance contracting appeared to be 
a promising means of systematically using incentives in school settings. 
The Office of Economic Opportunity (OEO) implemented a remedial education 
experiment in reading and mathematics. Federal support wks given to 18 
participating school districts for the subcontracting of remedial teaching 
to private educational technology companies. These contracts were of the 
performance contracting type, i.e., the private companies were to be 
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remunerated in proportion to the reading and mathematics achievement gain of 
the students they taught throughout the year* To provide a parallel assess- 
ment of performance incentive contracting without using a private educational 
technology company, OEO contracted with two school districts to test the use 
of incentives in regular classrooms, where teachers were remunerated in pro- 
portion to the achievement gains of their students (Ray, et at., 1972). 

At the same time, the RAND Corporation, under contract to HEW, performed 
an evaluation of performance contracting at five sites throughout the country* 
The products ^f this evaluation were: an analysis of the performance 
contracting concept (Stucker and Hall, 1971); case studies of eight perform- 
ance contracts (Carpenter, Hall, et al.^ 1972); and a performance contracting 
guide (Hall, et al.^ 1972)* These RAND documents were designed to provide - 
the educational community with a comprehensive and balanced appraisal of 
performance contracting* 

Also in 1971, American Institutes of Research of Palo Alto, California, 
conducted a feasibility study which included a review of the literature on 
incentives, the development of incentive models for field testing, the 
identification of candidate school districts to participate in an experiment, 
and the identification of cost estimates* In addition, AIR also conducted 
a pilot study for six weeks in one school district* With a strong caution 
about overgeneralizing from such a limited experience, AIR concluded that 
objectives-based incentive techniques* were effective in promoting student 
learning in basic skill areas* Jhis pilot study also led to several recom- 
mendations for refining the incentive models and for enhancing the accept- 
ance of the concept by parents and teachers (Jung, et al.j 1971; Lipe, et 
al., 1971)* 

The basic proposition of all these recent experimental incentives 
programs can be stated quite simply* Incentives focus motivation* 
Motivation, in turn, influences learning. Hence, the systematic application 
of incentives in the teaching-learning situation should lead to superior 
student achievement* Beyond this theoretical assertion, however, there 
remained a number of practical questions* How are the incentives to be 
supplied? What form should they take? To whom should they be offered? 
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In the study which is the object of this report, attention was focused 
not upon direct incentives to students but upon incentives to other partici- 
pants in the educational process, the teacher and the pdrent. In the case 
of the teachers, a traditional economic device was applied in a new context. 
A promise was made to pay a monetary bonus to the person charged with in- 
structing the children if he or she produced student achievement in excess 
of the predicted rate of growth. 

In the case of parents, a financial incentive was offered for their 
encouragement and support at home and at school to the extent that, as a 
result, their children attained a higher than predicted rate of achievement. 
Parent involvement is a recognized means of enhancing student performance, 
and the aim was to test the effectiveness of such a synergistic parent-teacher 
incentive model. 

Although the project had some classical experimental features, it must 
be recognized that the intent was not to conduct a formal, structured field 
experiment. There were uncontrolled variations throughout the experimental 
design. There were some program variations, i.e., departmentalization, types 
of books, eta.^ between experimental and control schools within districts. 
There were also program differences across the four cities. No attempt was 
made to specify classroom procedures or teaching techniques. Through a 
small scale, field pilot project, the purpose was to take a close look at the 
consequences of teacher and parent-teacher incentive models , to, learn 
more about the process by which they have their effect, and to test the 
reaction of the participants. In effect, this was a search for information 
and insights which could be used to make decisions on possible future 
experimentation in this area. 
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Objectives 



The basic purpose of the project was to study the effects of incentives 
offered to teachers or' to parents and teachers. There were two incentive 
models* In one, incentives were offered to teachers only in two cities* In 
the other, Incentives were offered to parents and teachers in two cities* 
Three major outcomes were to be considered: 

o Academic achievement of the students 

o Behavior of the parents, students, and teachers 

o Attitudes of the parents, students, and teachers* 

Implicit within this general statement of objectives are a number of 

specific questions, which can be divided into process, product, and policy 

categories* This report addresses primarily the product questions* The 

final reports by Education Turnkey Systems, Inc. (1972a L b) deal with 

process questions. Both reports consider policy questions* The specific 
questions that were addressed in the reports were: 

Product Questions 

o What influence did incentives have on student achievement gains 
in reading and mathematics, as measured by norm referenced tests? 

o VHiat effects did incentives have on student attitudes toward 

reading and mathematics, the school in general, and themselves? 

o Were there significant changes in academically related behavior 
of students attributable to the incentives program? 

o Were there specific changes in teacher behavior as a result of the 
offering of incentives? 

6 Did the offering of incentives produce changes in the nature or 

degree of parental support and encouragement of student achievement? 

o Did the incentives program produce changes in parental attitude 
toward their children's academic performance, the school system, 
or their children's teachers? 
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o With respect to all of the above, were there different effects 
attributable to the type of incentive model used? 

Process Questions 

o What factors influenced parent and/or teacher acceptance of, and 
participation in, the project? 

o What influences can be attributed to local and specific differences 
among teachers, parents, administrators, or other involved groups? 

0 What obstacles were encountered in implementing the incentives 
program or in making it operate successfully? 

Policy Questions 

o What impact did the incentives program have on teacher and parent 
organizations, school administrators, community groups, educational 
decision makers, and local politicians? 

o What changes should or could be made in the incentives models to 
increase student achievement in reading and mathematics? 

0 What changes should or could be made in the incentives models to 
make them more acceptable to professional, parental, and community 
groups or to eliminate specific undesirable features? 

b What trade-offs were feasible in cases where pupil achievement 
gains and teacher or community acceptance are at odds? 
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Project Organization 

Basic Features 

The project was designed and implemented as a field demonstration under 
the sponsorship of the Office of Planning, Budgeting and Evaluation (OPBE) at 
OE. It was monitored and evaluated by two independent contractors. The 
Planar Corporation was the Testing and Analysis Contractor (TAC) and was 
charged with the responsibility for achievement testing, attitude assess- 
ment and other matters relating to product evaluation. Education Turnkey 
Systems, Inc., the Monitoring and Assistance Contractor (MAC), was responsible 
for process evaluation and for assisting OPBE and the local school districts 
in the administration of the project. This report deals only with TAC 
activities. Separate process evaluation reports have been issued by MAC 
(Education Turnkey Systems, Inc., (1972a & b). The following summary of the 
division of responsibilities between MAC and TAC is provided to clarify the 
organizational structure and the working relationships between the evaluation 
contractors and the participating school districts. 

Two incentive models were employed in two cities each in this project. 
In two cities, incentives were offered only to teachers. Parents were 
informed of the project and were encouraged to support it, but they were 
not of fered any incentives for their support. In the other two cities^, 
both parents and teachers were offered incentives. In all four cities » 
control schools were selected, observed, monitored, and tested in exactly 
the same manner as the experimental school; however , incentives were not 
offered to the control schools. Hence, the only systematic differences 
were between experimental and control schools within cities and between 
incentive models in two. pairs of cities. 

The principal independent variable of the project was the of fer of 
incentive payments. The secondary independent variable was the type of 
incentive model: Parent-Teacher or Teacher Only. In nondepartmentalized 
grades, classroom teachers were offered cash bonuses of up to $1200 ($600 
for reading and $600 for mathematics) for- increasing the mean achievement 
level of their students. In departmentalized grades, teachers were offered 



up to §1200 divided by the number of classrooms he or she taught. In hoKh 
situations, qualification for the bonus was determined on the basis -^f each 
classroom's achievement gain above a predicted level, as determined by 
that classroom's pre to post-test differences on norm referenced reading 
and mathematics tests. The bonus was not an all-or-nothing proposition. A 
formula was devised to allow graduated payment for actual gains above 
expected gains. 

Incentive payments to the parent or guardian of students were offered 
on a similar, proportionate basis. Depending upon the pretest to post-test 
gain score of their child's classroom, the parent or guardian could receive 
up to §50 per child per subject. Thus, parents could obtain a maximum 
bonus of §100 for each of their children enrolled in the experimental school. 

The target population for the project was children in Grades 1 through 
6. A minimum of 80% were black children who were significantly below 
national norms in reading and mathematics. A high proportion of the children 
came from homes receiving some form of welfare assistance and from neighbor- 
hoods with low-income levels . These criteria were in keeping with the 
general aim of testing the incentive models as one means of addressing the 
chronic problem of the low-income underachieving student. 

Monitoring and Assistance Responsibilities 

The Monitoring and Assistance Contractor had the primary responsibility 
for making the initial protocol and working agreements with Federal, State 
and local officials and organizations. During the oroject, MAC maintained 
communication with the local school administrations, teachers, school board 
members, parents, and community representatives. MAC also monitored class-- 
room activities in the experimental and control schools at each site and 
provided feedback to OPPE ou the progress of the experiment. At the conclu- 
sion of the project^ MAC prepared two reports on process evaluation. The 
major tasks performed by MAC in each phase of the project were: 
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Setup Phase 

o Help negotiate with school districts for their participation and 
conclude agreements for payment of incentives . 

o Prepare and distribute explanatory materials and conduct orienta- 
tion sessions for project participants* 

o Assist TAG and OPPE in the development of the payment formula • 

o Develop a delivery system for the payment of earned incentives • 

o Prepare a monitoring plan and establish the mechanisms for project 
monitoring* 

Monitor and Assist Phase 

o Monitor classes and activities at experimental and control schools 

o Conduct continuing orientation and explanation of the project for 
teachers, students, parents, and other concerned groups* 

o Collect samples of test and curriculum materials in use in the 

participating schools and forward to TAC for the curriculum audit, 

o Report to OE on the progress of the project* 

Incentive Distribution, Analysis and Reporting Phase 

o Help to deliver incentive payments to teachers and parents, 

o Assist in arrangements for satisfactory completion of the project 
(submittal of necessary data and documents, debriefing of key 
participants, certification of incentive payment delivery, prepara* 
tion of news releases, etc*), 

o Conduct a cost analysis of the incentive models, 

o 'Prepare final reports describing project activities, explaining 

outcomes, and presenting conclusions on policy implications in the 
area of incentives* 



Testing and Analysis Responsibilities 

The following is a list of specific responsibilities of the Testing and 
Analysis Contractor. Activities carried out by TAG in support of these 
responsibilities and the results obtained are discussed in detail in subse- 
quent sections of this report. 

Setup Phase 

o Collect and analyze school district data to support site and 
school selection. 

o Design student, teacher, and parent questionnaires. 

o Research existing standardized achievements tests and ^recommend 
appropriate tests to OPPE. — 

o Assist OPPE in clarifying project outcomes, developing performance 
measures and designing appropriate instruments . 

o Design and maintain a biographic and background data bank for 
participating students . 

Operational Phase 

o Administer pre aud post-tests in reading and mathematics . 

o Design and develop a payment formula for teachers and parents 
based upon student pre-test scores in reading and mathematics. 

o Administer project related instruments to students, teachers, and 
parents. 

Analysis and Evaluation Phase 

o Score and analyze all project related instruments. 

o Analyze achievement scores in reading and mathematics. 

o Analyze attendance data. 

o Compute incentive payments for teachers and parents. 

o Audit curriculum materials used in experimental and control schools 
to ensure that test items were not being taught. 

o Prepare a final report evaluating project outcomes. 
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To carry out these responsibilities , TAC established a field organization 
at each of the four project sites. Data collection activities and adminis- 
trative arrangements with the schools were under the direction of a TAC Field 
Coordinator, who was assisted by a group of Field Investigators. This work- 
ing group conducted all on-site data collection for TAC and served as the 
nucleus of a much larger group of achievement test examiners hired locally 
to administer pre and post-testing of reading and mathematics achievement. 

At each site, the local school administration appointed a District 
Project Director drawn from the administrative or teaching staff, who acted 
as the on-site coordinator and liaison channel between TAC personnel and the 
principals and teachers of the participating schools. Figure I-l is a sche- 
matic representation of the project organization, showing the major lines of 
authority f rom OPBE down to the school level. It must be emphasized that 
Figure I-l indicates only organization structure and that, in practice, a 
close working relationship existed among OPBE, local school district, TAC, 
and MAC personnel. 
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Project Procedures 



Procedural Objectives 

Basically, this project was a "hands off" field demonstration project. 
There was no intervention by MAC or TAG in the conduct of classroom activi- 
ties or in the selection of instructional techniques employed by the teachers* 
Each teacher was completely free, within the limits of school policy, to use 
the method and/or materials of his or her choice to upgrade student perform- 
ance in reading and mathematics. The absolute amount and frequency of 
testing, observing and visiting were the same in the experimental and con- 
trol schools. Thus, the only experimental intervention was the incentive 
bonus offered in the experimental schools. 

In educational projects of this sort it is impossible to obtain a 
laboratory-like control of the situation. There are always many uncontrolled 
sources of variance, and no claim is made to have conducted a pure experiment. 
However, the basic strategy of nonintervention was an attempt to give the 
incentives as free a play as possible while observing the incentives' impact 
upon other elements of the school and community situation. The experimental 
strategy and the purpose of the demonstration can, therefore, be summed up in 
a single question: 

If we take two comparable schools with a large proportion 
of low-income, underachieving students and introduce special 
incentives in one school what differences between the schools 
will emerge and what are the research implication of these results? 

The procedures followed by TAG and MAG were conditioned by the basic 
policy of nonintervention. At the start of the project, MAG and TAG activi- 
ties were limited to conducting orientation sessions for the school 
personnel and parents and to drawing up the necessary working agreements. 
During the school year, MAG provided assistance, as required, in project 
administration and monitored activities in the experimental and control 
schools.,* TAG served as the agency for data collection and administered all 
achievement test instruments. Naturally, these activities drew attention to 
MAG and TAG by students, teachers, parents, and the community at large; but 
the basic posture was as nonobtrusive as possible. 
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Monitoring and Assistance Procedures 

The following sections describe the procedures to set up, administer, 
and monitor the project and to collect data for process evaluation. 

School Di strict and Teacher Agreements . MAC staff followed the Title 
III Guidelines and developed model proposals to be used for the Teacher-Only 
model and the Teacher-Parent model for the four school districts. Budgets 
were developed and discussed with each of the four sites to clarify total 
USOE commitment to the project and to identify the specific costs of per- 
sonnel and services at the individual sites . MAC continued to provide 
management support and technical assistance during the proposal development 
process from 24 August 1971 through 20 December 1971 at which time each 
site received notification that its Title III grant award was forthcoming. 

Participating sites were required to contract locally with the indivi- 
dual participating teachers and/or parents for the payment of incentives 
to be made at the end of the school year. MAC staff worked with the OE 
Project Officer to develop a model contract for teachers and a second con- 
tract for parents. The contracts were reviewed by administrative personnel 
at each site and revisions made to accommodate local administrative philo- 
sophies as much as possible without weakening the intent of the project. In 
several instances MAC staff worked directly with superintendents and school 
boards to arrive at an agreeable procedure by which the participating 
teachers could receive an incentive payment while under contract to the 
district to provide services as a teacher, and without violating the terms 
of" the local association/union agreement. 

This contract development process required seven revisions and modifica- 
tions before reaching a final agreement which was acceptable to all concerned 
parties. Presentations were then made at all project sites to explain the 
details of the contracts and the nature of the formula to be used in deter- 
mining the amount of incentives to be paid. This activity began on 29 July 
1971 and continued through April 1972 at which time the last teacher con- 
tract was signed at the Duval County, Florida, site. Parent contract nego- 
tiation continued through May 1972. 
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Monitoring and Assistance * Continuous on-site monitoring and adminis- 
trative assistance was the responsibility of MAC. The synopsis given below* 
is intended to outline the role of MAC and to indicate the procedures for 
monitoring and assistance • 

At the start of the school year, MAC prepared a "Standard Operating 
Procedures" manual for the school district Project Directors. The manual 
contained a description of the project, specification of procedures for 
administration and documentation, budgeting and accounting guidelines, and 
scheduling information. MAC also conducted a three-day workshop for 
Project Directors to orient them to the project, to review specific duties 
and responsibilities and to explain administrative procedures. 

MAC established and maintained a system for collecting monitoring in- 
formation and for serving as a clearinghouse for information about the con- 
duct of the project. MAC also prepared and distributed explanatory material, 
public information releases, and background documents for use by news media. 

Throughout the school year, the MAC representative at each site con- 
ducted a program of observations in the participating schools. This included 
observation of classroom activities, teacher and parent interviews, attend- 
ance at teacher and PTA meetings, and periodic meetings with school princi- 
pals and district administrative personnel. 

In concert with TAC, an audit of curriculum materials was performed 
by MAC. Items used in the classrooms were collected by MAC and forwarded 
to TAC to determine possible matching to specific items on the post-tests. 
The aim was to identify any potentially test biasing items, i.e. , class- 
room materials that were taken directly from the standardized achievement 
tests. Instances where such matching items appeared in classroom materials 
on other than a random or accidental basis were brought to the attention of 
OPPE. 

Testing and Analysis Procedures 

Data Collection, Testing and Analysis . TAC was charged with the major 
responsibility for data collection during the project. This included 
establishing and maintaining a basic biographic/demographic data file on 
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project participants, pre and post-testing of student achievement, class- 
room observations, and gathering information on attitudes and behaviors of 
students, parents, and teachers. With the exception of pre and post-testing, 
which required a large number of examiners hired and trained by TAC for that 
purpose, all data collection was carried out by the TAC Field Coordinator 
and a staff of four to six people working in cooperation with the school 
district Project Director. 

A variety of data collection procedures were employed, including group 
tests, questionnaires, interviews, observations, and excerpts of school 
district records. Table I-l is a summary of the methods used to collect 
data of each type for project participants. A brief description of these 
procedures is given below. A detailed discussion of the instruments, pro- 
cedures* and actual schedules is provided in the following chapter under 
the heading of "Measurement Techniques". 

TABLE i-l 
SUMMARY OF DATA COLLECTION METHODS 
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Achievement Tests > The Metropolitan Achievement Tests, 1970 edition 
(MAT) were used as pre and post-tests in reading and mathematics and were 
administered to all project students in the control and experimental schools 
by school district substitute teachers trained and supervised by TAG. Home- 
room teachers were present in the classroom but did not take part in the 
testing. Pretesting was carried out within the first three weeks after the 
start of school in each city, and post-testing was conducted three weeks prior 
to the end of the school year* Students entering both schools during the 
school year, up to the last month before post-testing, were also pretested. 
The data from these new students were analyzed separately to determine the 
impact of incentives on- achievement rates over periods of time shorter than 
a school year. " 

Questionnaires . Questionnaires to elicit attitudes and to obtain 
self-reports of behavior were administered to all experimental and control 
participants twice during the year. Student questionnaires were administered 
in the classrooms by TAG examiners. At the same time, parent questionnaires 
were given to students to be taken home. Responses were mailed by the parents 
to the TAG project office in Washington, D,G.. The parent questionnaire Jealt 
with attitudes, self -reports of behavior (specific assistance given to chil- 
dren, study facilities in the home, attendance at school meetings), and family 
demographic information. For both pre and post parent questionnaires, 
nonrespondents were randomly sampled and were sent a follow-up questionnaire. 
If the selected parents did not return the follow-up questionnaire, they were 
telephoned at home and requested to complete and mail it. 

The teacher questionnaires were distributed through the schools and 
returned to the TAG project office in Washington by mail. In addition to 
attitudinal atid behavioral information, the questionnaire contained items 
relating to teacher background, experience, and qualifications. All ques- 
tionnaire responses from all participants were filed by a project-unique 
identification number that guaranteed the respondent's anonymity. 

Interviews ♦ All teachers in the experimental and control schools 
were interviewed by TAG field personnel twice during the year, in the fall 
and spring terms. In addition to questions pertaining to attitude and 
specific elements of teaching practice, free response questions were * included 
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to elicit a broad range of opinions on incentives, the project itself, the 
students, the school, and the teaching profession. The free response items 
were analyzed by OE and MAC as part of the process evaluation. All teacher 
interviews were conducted in the school. 

Fifty parents and 50 students per city were interviewed in the fall 
and spring by TAC field personnel. For each control or experiinental school 
a stratified random sample of parents by homeroom was selected. To assure 
full and equitable coverage across all schools and sites, a i00% oversample 
was drawn. Parents were contacted and interviewed in their homes until a 
total of 25 interviews had been completed for each school. Care was taken 
to preserve an even distribution of interviews across homerooms. Tn effect, 
this amounted to two or three parent interviews per homeroom depending upon 
the size of the school. After the parents had been interviewed, arrange- 
ments were made through the teachers to interview the children of these 
parents during school hours. Thus, a matched set of parent-student responses 
was obtained. The post-test interviews were conducted with the same parents 
who had been interviewed in the fall. In those cases where the parents were 
not available for post-interviews, another name was selected from the same 
homeroom and then the parent's child was also interviewed. Free response 
items on the parent and student interviews were also analyzed by OE and MAC 
as part of the process evaluation. 

Observations. TAC representatives conducted two series of class- 
room observations during the school year. Each classroom was visited for four 
20-minute periods in November-December 1971 and again for four 20-minute 
periods in March-April 1972. Observers recorded teacher and student behavior 
according to a uniform schedule. Data were collected only on observed 
teacher and student behavior; no attempt was made by the observer to record 
attitudes or to draw inferences as to the meaning of classroom activities. 

Extracts of School Records. School district Project Directors, 
with the assistance of local TAC field personnel, compiled a basic biographic/ 
demographic record for each student in the project. This involved reviewing 
and excerpting school records to obtain such information as age, sex, address, 
siblings in the same school, years in the school, and so on. This was done 
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for each student in the control and experimental schools at the start of the 
program and for transfers when they entered either school. Supplementing 
this 9 a record was kept of attendance throughout the school year. 

Table 1-2 is a summary of data collection activities during the project. 
Each data collection event is listed according to the scheduled time of per- 
formance. However, it- must be eiq>hasized that the schedule shown here is 
only a nominal schedule and that certain deviations and delays were imposed 
by local conditions and unforeseen circumstances. The specific reasons for 
these departures from schedule will be taken up in the next chapter in the 
discussion of how each instrument was administered. 
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Payment Formula 

The original intent of the project was to employ a siraple incentive 
formula whereby the minimum achievement gain for incentive payments, the 
Base Gain Indicator (BGI) , would be calculated by subtracting the mean pre- 
test grade equivalent score for a given subject and grade level from the 
corresponding pretest score for the next higher grade level. That is, 

BGI « X . - - X 
n + 1 n 

Where BGI « Base Gain Indicator 

X^ « mean pretest grade equivalent score for a given grade (n) 

\ + 1 * pretest grade equivalent score for the next higher 

grade (n + 1). 

Thus, if the mean grade equivalent pretest score in reading for the second 
grade of a particular school were 1.8 and the corresponding third grade 
score were 2.6, second grade students in a given classroom would have to 
gain an average of 0.8 grade level on the post-test for their teacher to 
qualify for the minimum incentive payment. 

As a result of conferences with local school officials and teachers 
during and after the pretesting, TAC and OPBE concluded that such a proce- 
dure might be inherently unfair to teachers with low achieving students in 
homogeneously grouped classrooms. To determine the extent to which homo- 
geneous achievement grouping actually existed in the experimental schools, 
TAC conducted an analysis of the pretest scores of all reading and mathe- 
matics classes in the experimental schools. Using the chi-square test with 
the .01 level of significance as a criterion, the individual classroom 
groups were found to be significantly different in 37 of 48 cases (A schools 
X 6 grade levels X 2 subjects). Thus, in over three-quarters (77%) of the 
classrooms, it could be assumed that students were stratified according to 
achievement level and that in a sizable proportion of these cases the con- 
centration of low achievers would operate to the disadvantage of the teachers 
in attaining incentive payments. 
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Derivation of the Base Gain Indicator > For this reason, it was decided to 
derive a payment formula which would be sensitive to the initial achievement 
level (and, hence, the potential for gain) for each classroom. As a first step, 
the student population in each grade in each school was rank ordered twice; 
first, by MAT standard scores in reading, and then by MAT standard scores 
in mathematics* Standard scores were used Instead of raw scores because 
there were as many as four different test levels used within a grade and 
raw scores are not comparable among test levels. 

The next step was to divide each grade level in each experimental 
school into thirds (high, middle, and low achievers). The division point 
between one achievement level and t:he next was at that standard score 
which was closest to one-third of the total grade level for that school 
population. This procedure resulted In groups which were of slightly unequal 
size, but it assured that all students with the same standard score were 
placed in the same group. 

Next, the means for each of the three groups in each subject were cal- 
culated, yielding six means for each grade level, three for reading and 
three for mathematics. Because there were instances of severe anomalies 
between the means for successive grade levels, it was necessary to calculate 
regression equations for the high, middle, and low reading and mathematics 
groups across the six grade levels. By then fitting a regression curve to 
the means, the base gain indicators were smoothed and no longer subject to 
inordinate variations from one grade level to the next. Several different 
regression equations were tried, but the one which produced the best fit 
across all sites. Grades 1 through 5, and achievement groups was a second 
order polynomial: 

L = a + bG + cG^ 
P 

Where L « predicted level of student achievement expressed as a 
^ grade equivalent score 

G » present grade level 

a,b, and c « empirically derived coefficients. 
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Table 1-3 presents the predicted grade equivalent scores derived from the 
regression equation. Because this regressiorTformula predicted post-test 
scores for Grade 6 that were significantly greater than could be reasonably 
expected, a linear regression equation was used for that gradri level. 

As a final step, a base gain indicator was derived. The BGI is the least 
amount of gain that a classroom had to achieve for the teacher to obtain a 
bonus payment. It was calculated for a given grade level by subtracting the 

classroom pretest score for a given achievement group from the 
predicted pretest score for the same achievement group (high, medium, or 
low) at the next higher grade level. 

Thus: 

BGI « L - L 

Pn + 1 Pn 

Where BGI « Base Gain Indicator 

L « predicted level of student achievement (in grade equivalent 
*^n score) for a given grade (n) 

* predicted level of student achievement (in grade equiva- 
n + 1 lent score) for the next higher grade (n + 1). 

Thus, for reading in Grade 2, for example, there would be three BGIs, "one 
each for the high, middle, and low achievement groups. Table 1-4 lists the 
BGIs for each site, grade, achievement group, and subject. 

To determine which BGI to apply to a given classroom teacher, the 
average pretest score for the individual classroom was computed for each 
subject. Then it was determined which achievement interval (high, middle, 
or low) that classroom fell within. The BGI for that achievement group 
was then assigned to the classroom. 

Teacher Payment Formula . A graduated scale of four payment incre- 
ments was established for the. teachers. To obtain a minimum payment for one 
subject (25% of the $600 maximum, or $150), the teacher of a given self- 
contained class had to raise the students' mean achievement level, as 
measured by the post- test, by an amount equal to the BGI. For each 0.1 
grade equivalent above the BGI, the teacher obtained an additional $150. 
The maximum bonus of $600 per subject was thus attained if the mean 

(Teat veemee on page 1^28) 
I'll 



\ TABLE 1-3 

t 

PREDICTED MEAN GRADE EQUIVALENT SCORES FOR EXPERIMENTAL GROUP STUDENTS 







Reading 






Arithmetic 




Oakland 


High 


Medium 


Low 


Oakland 


High 


Medium 


Low 


1 


1.67 


1.41 


1.27 


1 


1.36 


1.18 


1.02 


2 


2.12 


1.59 


1.34 


2 


1.98 


1.48 


1.13 


3 


2.86 


1.99 


1.54 


3 


2.73 


1.97 


1.40 


4 


3.89 


2.65 


1.87 


4 


3.74 


2.67 


1.84 


5 


5.21 


3.39 


2.32 


5 


4.96 


3.58 


2.45 


6 


6.79 


4.41 


2.90 


6 


6.38 


4.68 


3.23 


7 


8.41 


5.61 


3.60 


7 


8.02 


5.92 


4.17 


Cincinnati 








Cincinnati 








1 


1.43 


1.28 


1.23 


1 


l.lfe 


1.05 


0.94 


2 


2.13 


1.69 


1.42 


2 


2.11 


1.45 


1.19 


3 


2.88 


2.12 


1.65 


3 


2.99 


1.96 


1.52 


4 


3.68 


2.57 


1.90 


4 


3.82 


2.57 


1.93 


5 


4.54 


3.04 


2.19 


5 


4.57 


3.29 


2.42 


6 


5.45 


3.53 


2.50 


6 


5.27 


4.10 


3.00 


7 


6.40 


4.03 


2.83 


7 


5.90 


5.02 


3.66 


San Antonio 






San Antonio 








1 


1.41 


1.27 


1.20 


1 


1.28 


1.08 


0.94 


2 


2.12 


1.64 


1.40 


2 


1.99 


1.47 


1.15 


3 


2.82 


2.01 


1.61 


3 


I'.n 


1.95 


1.43 


4 


3.52 


2.39 


1.83 


4 


3.63 


2.52 


1.79 


5 


4.23 


2.78 


2.06 


5 


4.55 


3.19 


2.21 


6 


4.93 


. 3.17 


2.30 


6 


5.54 


3.94 


2.70 


7 


6.33 


3.57 


2.60 


7 


6.60 


4.79 


3.06 


Jacksonville 






Jacksonville 






1 


1.39 


1.21 


1.17 


1 


1.19 


0.96 


0.87 


2 


1.69 


1.43 


1.28 


2 


1.52 


1.19 


0.99 


3 


2.21 


1.69 


1.41 


3 


2.12 


1.60 


l.?l 


4 


2.92 


2.01 


1.56 


4 


2.98 


2.18 


1.53 


5 


3.83 


2.37 


1.73 


5 


4.11 


2.93 


1.96 


6 


4.95 


2.78 


1.91 


6 


5.50 


3.85 


2.48 


7 


6.27 


3.24 


2.12 


7 


7.15 


4.95 


3.09 



ERIC 



TABLE 1-4 
BASE GAIN INDICATORS— CINCINNATI 



GRADE LEVEL 


ARITHMETIC ACHIEVEMENT LEVEL 




Low 


Meditm 


High 


1 


.25 


.41 


.95 


2 


.33 


.51 


.89 


3 


.41 


.61 


.82 


4 


.49 


.71 


.76 


5 


.58 


.81 


.70 


6 


.66 


.91 


.63 



GRADE LEVEL 


READING ACHIEVeiENT 


LEVEL 




Low 


Meditm 


High 


1 


.20 


.41 


.70 


2 


.23 


.43 


.75 


3 


.25 


.45 


.80 


4 


.28 


.47 


.86 


5 


.31 


.49 


.91 


6 


.34 


.51 


.96 



1-24 



TABLE 1-4 (Continued) 
BASE GAIN INDICATORS—JACKSONVILLE 



GRADE LEVEL 


ARITHMETIC ACHIEVEMENT LEVEL 




Low 


Medium 


High 


1 


.12 


.23 


.34 


2 


.22 


.40 


.60 


3 


.32 


.58 


.86 


4 


.42 


.75 


1.13 


5 


.52 


.92 


1.39 


6 


.62 


1.10 


1.65 



GRADE LEVEL 


Low 


READING ACHIEVEMENT LEVEL 

Medium High 


1 


.11 


.22 


.31 


2 


.13 


.26 


.51 


3 


.15 


.31 


.71 


4 


.17 


.36 


.92 


5 


.19 


.41 


r.i2 


6 


.21 


.46 


1.32 
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TABLE 1-4 (Continued) 
BASE GAIN INDICATORS— OAKLAND 



GRADE LEVEL 


ARITHMETIC ACHIEVEMENT LEVEL 




Low 


Medium 


High 


1 


.10 


.30 


.58 


2 


.27 


.50 


.79 


3 


.44 


.70 


1.00" 


4 


.61 


.90 


1.21 


5 


.76 


1.10 


1.42 


6 


.94 


1.30 


1.64 



GRADE LEVEL 


READING ACHIEVEMENT LEVEL 




Low 


Medium 


High 


1 


.07 


.18 


.45 


2 


.20 


.39 


.74 


3 


.33 


.60 


1.03 


4 


.45 


.81 


1.32 


5 


.58 


1.01 


1.60 


6 


.71 


1.22 


1.89 
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TABLE 1-4 (Continued) 
BASE GAIN INDICATORS—SAN ANTONIO 



GRADE LEVEL 


ARITHMETIC ACHIEVEMENT 
Low Medium 


LEVEL 
High 


1 


.21 


.39 


.71 


2 


.28 


.48 


.78 


3 


.35 


.57 


.85 


4 


.42 


.66 


.92 


5 


.49 


.75 


.99 


6 


.56 


.84 


1.10 



GRADE LEVEL 


READING ACHIEVEMENT 
Low Medium 


LEVEL 
High 


1 


.20 


.37 


.70 


2 


.21 


.37 


.70 


3 


.22 


.38 


.70 


4 


.23 


.39 


.70 


5 


.24 


.39 


.70 


6 


.25 


.40 


.70 
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achievement gain of the. class was 0.3 grade equivalent or more above the BGI. 
If the class failed to attain the BGI, the teacher received no bonus. Table 
1-5 shows the incentive payment schedule in relation to achievement gains. 



TABLE 1-5 

TEACHER INCENTIVE PAYMENT SCHEDULE 
FOR SELF-CONTAINED CLASSROOMS 



Mean Post-Test 
Achievement Gain 


Teacher 
Incentive Payment 


<BCI 


$0 


BGI 


$150 V total number of 
classrooms taught 
that subject 


BGI + 0.1 G.E. 


$300 ± total number of 
classrooms taught 
that subject 


BGI + 0,2 G.E, 


$450 ^ total number of 
classrooms taught 
that subject 


BGI + 0.3 G.E. 

' 


$600 i- total number of 
classrooms taught 
that subject 



It should be remembered that the teacher was eligible for an incentive 
payment in both reading and mathematics. If, for example, the mean class 
gain in reading were BGI + 0.1 and in mathematics BGI + 0.2, the teacher 
received a bonus of $750. In departmentalized grade levels, the maximum 
incentive per classroom was $1200 divided by the number of classrooms taught 
by that teacher. The payment schedule was then adjusted to provide payment 
jp'v'c-'^s vithin that maximum. 

rj^L^J^i^T Payment F ormula . The schedule of incentive payments to 
parents was similar to that of teachers. Parent payment was dependent upon 
the performance of their child's classroom, not the child's individual 
achievement. The maximum incentive bonus to parents was $50 per subject. 
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payable in increments of $12.50 according to the schedule in Table 1-6. 



TABLE 1-6 

PARENT INCENTIVE PAYMENT SCHEDULE 
FOR READING OR MATHEMATICS 



Child's Classroom 
Achievement Gain 


Parent Incentive 
Payment 


>BGI 


$0 


BGI 


$12.50 


BGI + 0.1 G.E. 


$25.00 


BGI + 0.2 G.E. 


$37.50 


BGI + 0.3 G.E. 


$50.00 



Incentives were paid to parents for each child in the experimental 
school. A parent with two children, with each child's classroom making a 
gain of 0.3 grade equivalent above the BGI in both reading and mathematics, 
would therefore qualify for a bonus of $200. 
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Project Sites 

The incentives experiment was conducted at schools in four cities: 
Cincinnati, Ohio; Jacksonville (Duval County), Florida; Oakland, California; 
and San Antonio, Texas • These four sites were selected by OE from a list of 
school districts identified by OE as potential participants. The site and 
school selection process, which was a joint activity by MAC and TAG under 
the direction of OPBE, is described below, 

' Site Select ion 

At the outset of the project, a list of candidate school districts was 
drawn up by OE, The following criteria were used to determine initial eli- 
gibility: 

o The school district had to be in a medium or large city, popula- 
tion between 250,000 and 2,000,000, 

o The school district had to include large absolute numbers of 

black children, who represented a high percentage of the school 
population, 

o The school district had to include large numbers of children from 
low-income families, on both an absolute and proportional basis, 

o The school district had to be in a state which had unobligated 
ESEA Title III, Section 306 funds (Commissioner's Discretionary 
Fund) available at the start of the project. 

The criterion of medium-city size was finally adopted for two reasons. 
First, it was consistent with the overall project aim of testing incentives 
models in an urban setting. Second, by excluding major metropolitan areas 
in favor of medium-sized cities, it was possible to avoid administrative 
and logistical problems inherent in very large school districts and, at the 
same time, to assure that there would be a sufficient number and concentra- 
tion of minority, poor students. The second and third criteria follow 
logically from the basic objective of demonstrating incentives as a possible 
approach to helping underachieving, low- income students. The fourth 
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criterion (availability of Title III funds) was included for administrative 
reasons. The funds to pay school district administrative costs for the 
project and to pay for incentives earned by teachers and parents were drawn 
from ESEA Title III, Section 306 (Connnissioner 's Discretionary Fund).* 

More than 50 school districts were determined by OE to be eligible 
according to these five criteria. Telephone and mail inquiries were made 
to these eligible districts by MAC under the direction of OPPE to ascertain 
their willingness to participate in the project. Eleven districts indicated 
that they wished to be considered and visits were made to all 9f these 
candidate sites by OPBE, MAC and TAC representatives. The final selection 
of the four participating districts was on the basis of: the degree of the 
district's desire to participate in the project; the ability of the district 
to obtain cooperation from the local teacher organization; the likelihood 
of favorable community attitudes; and the availability of schools meeting 
specific criteria to serve as either experimental or control schools. 

Table 1-7 is a summary of the characteristics of the participating 
local school districts in terms of the objective selection criteria. The 
table also indicates which incentive model was selected by each district. 
A tentative decision on the incentive model was made at the time the site 
was selected. The final decision was based upon the preference of the 
school district administration and the school board. This requirement that 
local preferences prevail entailed a change from the original assignment 
of a model in Cincinnati. The school administrators in that city had ori- 
ginally accepted the Parent- Teacher model; however, prior to the pretesting, 
the school board decided to accept the Teacher- Only model. Fortunately, 
the San Antonio school board was willing to change from the Teacher-Only to 
the Parent-Teacher model, thereby preserving the basic design of the project. 
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TABLE 1-7 

CHARACTERISTICS OF PARTICIPATING SCHOOL DISTRICTS 



SELECTION CRITERU 




PROJECT 


SITES 






Jackson- 

^v4 11a 

vixxe 


Oakland 


San 
Antonio 


Population^ 


452,524 


528,865 


361,561 


654,153 


Elementary School Enrollment^ 


50,206 


54,986 


35 950 


HX,UO/ 


2 

Number Minority Children 


22,365^ 


20,316^ 


26,251^ 


33,396^ 


Percent Minority Children^ 


44.5^ 


30.7^ 


73.0^ 


81.3^ 


Number Poor Children^ 


11,388 


18,090^ 


10,472^ 


14,003 


Percent Poor Children^ 


22.7 


32.9 


29.1 


34.1 


Title III Available 


Yes 


Yes 


Yes 


Yes 


Incentive Model 


Teacher 


Teacher 


Parent 


Parent 




Only 


Only 


Teacher 


Teacher 



1. Based on 1970 census figures • 

2. Based on school enrollment 1970-71. 

3. Black children only. 

4. Includes Black, Spanish American, Oriental, American Indian, and other 
nonwhites . 

5. According to Title I poverty criteria. 

6. Proportionate estimate based on total school enrollment. 



School Selection 

The selection by the OE Project Officer of individual schools eligible 
to participate in the project was based upon the following considerations: 

o The school was participating in an ESEA Title I program or, if 
not participating, was eligible for Title I funds. 

o The school had at least 80% black children from poor homes. 
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o The school had Grades 1 through 6 • 

o Tlie school was not participating in Federally supporced instruc- 
tional or remedial programs other than ESEA Title I. 

,0 'The school had a demonstrated record of underachievement , as 

measured by recent standardized tests in reading and mathematics* 

The schools in each district were ranked by TAC in terms of these five 
criteria and then grouped into pairs , matched as closely as possible to 
ensure comparability between experimental and control groups. Each pair 
of candidate schools was then screened by a group consisting of OPPE, MAC, 
TAC, and district administration members to make a final choice. Among the 
characteristics which led to selection of the pair of schools to participate 
in the program were: 

o Achievement test scores below the other pairs 

o A high percentage of low-income students 

o The likelihood of support by the school principals 

o The likely acceptance of the program by the teaching staff, 
school parents^ and the immediate communities. 

Whenever possible, as part of the final selection process, interviews 
were conducted with principals and teacher representatives. Specific points 
of discussion were: the willingness of the principal and teachers to 
cooperate in the project, and assurances by the school district that the 
teaching staff from the previous year would be retained, except for those 
who already had an approved commitment for transfer. The experimental 
school was chosen on the basis of its having lower academic performance and 
more children from poverty homes than the control school. 

The experimental and control schools finally selected for participation 
in each city are listed below: 

Cincinnati 

Garfield Elementary School (experimental) 
Cummins Elementary School (control) 
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Jacksonville 

Livingston Eleaentary School (experimental) 
Carver Elementary School (control) 

Oakland 

Longfellow Elementary School (experimental) 
Santa Fe Elementary School (control) 

San Antonio 

Washington Elementary School (experimental) 
Tynan Elementary School (control) 

Table 1-8 on the following page is a summary of the student population 
characteristics in the experimental and control schools. Note that two 
control schools are listed in Oakland. A group of teachers in the control 
school originally selected, Washington Elementary School, decided at the 
start of the school year to withdraw from the project. A suitable replace- 
ment, Santa Fe Elementary School, was arranged just prior to the scheduled 
start of pretesting. Both schools are listed in Table 1-8 for the sake 
of comparison, but it should be understood that only data from Santa Fe were 
used in determining project outcomes in Oakland. 

Table 1-9 contains student achievement test results in reading and 
mathematics for 1970-71, the year before the start of the project. Results 
are tabulated under the grade level of the students in the year in which 
tested. Scores are reported both in grade equivalent form (in the column 
titled G.E.) and in terms of grade level decrement (in the column titled 
DEC). The grade level decrement is an approximate index of how far the 
schools were behind national norms. Thus, if the mean achievement test 
score for fourth grade students tested in October was 3.5, the decrement is 
listed as -0.7, which represents the difference between what they should 
have scored on the second month of the fourth year (4.2) and what they 
actually scored (3.5). Grades in which tests had not been administered by 
the school districts are denoted by the entry "NO". 
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TABLE 1-8 

CHARACTERISTICS OF PARTICIPATING SCHOOLS 



SCHOOL 


No.^ 
Students 


Mo. 
Black 


% 

Minority 


M 2,3 
No. * 

Poor 


2 3 

Poor 


TITLE I 


Ellg. 


Redp. 


uxncmnaci 
















KrOT r le Xu \tt) 


612 


576 


Z7*t . X 




vX. U 


Y 


Y 


vuimnxns 










.J 


Y 


Y 


Jacksonville 
















Lxvxngs con v '^j 




oo/ 


99.8 




5o.7 


Y 


Y 




iniQ 

XUX7 


ini ft 

XUXO 








Y 


Y 


Oakland 
















Longfellow (E) 


608 


592 


97.4 


258 


42.4 


Y 


N 


Washington (C)^ 


611 


591 


96.7 


310 


50.7 


Y 


N 


Santa Fe (C) 


856 


836 


97.7 


503 


58.8 


Y 


N 


San Antonio 
















Washington (E) 


654 


620 


94.8 


236 


36.1 


Y 


Y 


Tynan (C) 


429 


418 


97.4 


181 


42.2 


Y 


Y 


1. Withdrew at start of project, replaced by Santa Fe. 

2. 1970-71 enrollment figures. 

3. Family meets Title I pov€frty criteria. 
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CHAPTER II 
METHODOLOGY 

Introductiori 

Although the project was a field demonstration and not a formal 
experiment 9 the selection or development of measurement instruments and 
their administration were guided by formally stated evaluation objectives. 
A process was followed in which desired outcomes were stated as hypo- 
theses » items were developed (or instruments selected in the case of 
achievement outcomes) and then the items were assigned to the instruments. 

There were two objectives in this activity. The first objective was 
to ensure that all parties involved in the design of the project had an 
opportunity jointly to present their views about the outcomes. Too often 
in the design of large scale projects or field evaluations » because 
there are many uncontrolled sources of variance Imd because several groups 
participate in the design, outcomes are not treated systematically or 
coiiq>rehensively during the planning phase. Consequently » there are gaps 
in the data and many unanswered questions at the conclusion of these 
projects. The instrument development process used in this project was 
designed to avoid these inadequacies. The second objective was to ensure 
that items or questions developed to measure the same hypotheses were 
assigned to different instruments. Because interviews , questionnaires , 
and observations were to be used by both MAC and TAC, it was desirable 
that the reliability of the measurement be increased by spreading these 
items out over the severrl different types of data gathering activities 
employed by the two contractors. There were ten different data collec- 
tion instruments developed and used by TAC and MAC in this project. 
Ranging from inventory checklists to open-ended interview protocols » 
these instruments provided a unique opportunity to distribute items 
related to the same variable among a wide variety of instruments. 
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Obviously, the systematic development or selection of instruments is 
the necessary but not sufficient first step for an adequate evaluation of 
any program. The specificity of the questions, the sensitivity of the 
data collection techniques, the appropriateness of the analysis and the 
intelligence of the interpretation are critical intervening variables 
between this development process and the final report. The variable and 
instrument development process followed in this project was important only 
in its power to facilitate the performance of subsequent tasks and deci- 
sions. 
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Achievement Measurement 



Instrument Selection 

Achievement Specifications . Four specifications were established by 
OE pertaining to the measurement and evaluation of student achievement. 
Two of these were outcome specifications; one was procedural; and one 
was an evaluation strategy specification. The first outcome specification 
was that only Reading and Mathematics achievement were to be measured. 
Schools teach many skills and attitudes, but Reading and Mathematics 
were selected for attention because of the discouraging pattern of low- 
income school results in these two areas. Furthermore, because this 
project was as much a demonstration of the logistical feasibility of 
offering incentives as it was an experiment, Reading and Mathematics were 
defined in the narrowest sense of the two terms. Consequently, for 
example, student language arts and spelling skills were not to be con- 
sidered. It was believed that by focusing upon a narrow range of skills, 
the issue of the practical feasibility of incentives could be clearly 
addressed in this field demonstration. 

Procedurally, because the tests were to be used to determine pay- 
ments as well as to evaluate the impact of the offer of incentives, it 
was necessary to guarantee that there was no opportunity to "teach the 
test". Since the time of the allegations about teaching the test in 
the Texarkana, Arkansas, and Providence, Rhode Island performance con- 
tracts, any project that involves payment for results, whether it is 
performance contracting or teaching incentives, must account for this 
factor. Multiple testing (one test for payment and cr.e test for evalua- 
tion) with both tests disguised or masked has been the usual complete 
solution to this problem. However, its cost in terms of time, student 
patience and teacher good will are formidable practical barriers. OE 
was aware of these conflicting considerations and had specified that the 
achievement assessment in this project be designed to resolve them. 
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Because this project was conceived and administered at the Federal 
level, the paramount evaluation objective was that the achievement results 
be measured in clear and understandable terms* The Federal Government 
has many audiences, and it was believed that the achievement results of 
this project should be based upon instruments and analyses acceptable to 
both educational researchers and practitioners* This requirement meant 
that the achievement tests meet high psychometric standards and yet not 
be so arcane as to be uninterpre table by those who are not research 
specialists. In short, it was important that the educational and statis- 
tical meanings of this project be considered in the selection .nd subse- 
quent analysis of the achievement tests. 

Test Selection . The Metropolitan Achievement Test Batteries, 1970 
Edition, (MAT), published by Harcourt Brace and Jovanovich, Inc., were 
selected for administration after a careful examination of a variety of 
measurement instruments and other factors. In view of the many valid 
criticisms of performance contracting instrumentation and analysis, the 
decision to select the MAT series was not taken lightly. In arriving 
at these decisions, the following factors were considered: 

o Because of the populations in this project, it was decided that 
test batteries should be assigned individually to each student 
on the basis of the best available information about his or 
her current Reading achievement level. This meant that the 
achievement test levels would be mixed within grade levels 
and that these mixes would vary between experimental and con- 
trol schools within sites and among the four sites. Therefore, 
the achievement tests had to have a score scale that would 
allow comparisons across test levels for evaluation and pay- 
ment gains analysis. The results of the MAT, Primer through 
Intermediate Batteries, are expressed for all batteries and 
all forms of those batteries on a single, common scale. This 
made the MAT uniquely suited to the assignment of tests on an 
individual basis. 
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Because this project did not Impose any materials or methods 
upon the teachers » the achievement test content had to be 
appropriate to the wide range of curriculum materials that 
would be used throughout the project* In addition, because of 
the rapid curricula changes in the past few years » it was neces- 
sary that the achievement tests have recently developed editions 
so that the content validity would be high. The MAT had under- 
gone an extensive content revision in 1970. 

Because the participants in the project were low-income, inner- 
city students, it was essential that the norming population 
be representative of the demographic trends in the nation. 
With the urbanization of the population, and with low-income, 
inner-city children increasing as a percentage of public 
elementary students, the achievement tests had to be normed 
on a representative and recent sample. The MAT had been re- 
standardized in 1970. 

The tests had to meet acceptable psychometric validity and 
reliability standards. The MAT has separate fall and spring 
forms and norms. 

This project was seen as part of a continuum of Federal 
research and experimentation in education. The OEO perform- 
ance contracting experiment and die OE Anchor Test study were 
important elements in that continuum, and it was desirable 
that the results of this project be comparable to those of at 
least these other two major efforts. The MAT was used for 
the evaluation of the OEO experiment and was the /'anchor test'' 
in the OE Anchor Test study. 

Finally, there were a n^rriad of practical considerations. 
Because the project involved widely separate and disparate 
sites, there had to be at least two alternative forms. The 
format and administrative procedures had to be appropriate 
to the populations of interest. And the scoring facilities 
had to be convenient. The MAT series was highly satisfactory 
in all these respects. 
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Test Description 

Norm Data, The Primer, Primary. I, Primary II, Elementary and Inter- 
mediate levels of the Metropolitan Achievement Tests were selected as the 
test series for this project. This series was one of the most recently 
revised in terms of content and norm data. The salient norm data for the 
MAT standardization group and for the project populations are presented 
in Table II-l. As can be seen from the table, the MAT standardization 
population is quite representative of the national population; approxi- 
mately 44,000 students in Grades 1-6 participated In the MAT standardiza- 
tion. 

Although the recent MAT normlng was representative of the national 
population and certainly comprehensive, it must be emphasized that the 
student population in this project was a subpopulatlon, not identical 
with the total norm group. The figures for the project population pre- 
sented in Table II-l are based upon questionnaires sent to all of the 
parents in March and April of 1972. Out of a possible return of 4500 
questionnaires, only 1329, or 31%, were received. Thus, it is clear that 
generalizations from the respondents to the total population should be 
made with great caution. From an examination of the two populations in 
Table II-l, it can be seen that there are vast differences between the 
project population and the norm populations. Because of the urbanlsm, 
low Income, and minority representation found in the project population, 
there are obvious limitations to the appropriateness of the tests' con- 
tent. The children in the project come from a highly distinctive back- 
ground. The content of the tests and, perhaps more Importantly, the 
testing procedures themselves, were not designed with these children In 
mind. 
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TABLE II-l 



COMPARISON OF STUDENT POPULATIONS, 
PROJECT (1972), MAT STANDARDIZATION (1970), 
SELECTED DEMOGRAPHIC CHARACTERISTICS 



DEMOGRAPHIC 


PROJECT , 


MAT 2 


CHARACTERISTIC 


POPULATION 


POPULATION 


City size 






- 250,000 


100% 


21% 








- 2,500-24,999 




30% 


- Less than 2,500 




27% 


Family income 


$3639 


$5550 


Minority Group 






- Black 


95% 


13% 


- Spanish-speaking 


3% 


4% 


- Oriental 


.1% 


4% 


- American Indian 




.3% 


Years of schooling 


12.2 


10.7 


H.S. gradtiates, parent 


44% 


41% 


Unemployed 


26% 


5.1% " 



Sources: 

1. Incentives Project Parent Questionnaire. 

2. Harcourt Brace Jovanovich, Inc., Report No. 7, June 1971. 



Despite these llmitatioxis of content and the reliance upon only one 
type of test- taking skill, the use of standardized tests such as the MAT 
is justifiable for evaluation and payment in a project of this kind. The 
content of the Reading and Math test particularly reflects the kinds of 
skills which the public schools strive to develop in all pupils. These 
skills, moreover, are demonstrably essential to success in later schooling 
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and in most occupatioiis • Also, there is really no feasible alternative 
to the use of such tests for a project of this kind. The standardized 
tests are relatively sophisticated and adequate instruments for assessing 
the performance of children in terms of a generally accepted curriculum 
content and with the use of widespread teaching practices. . To have 
attempted to develop and use some other form of test instniment would 
have been an endeavor beyond the scope of this or any other demonstration 
project. 

Finally, one other point which makes the use of these tests defensible 
in this project is that the results obtained from the children in the in- 
centives schools are not being compared with national norms, but with 
the results obtained from quite similar children in quite similar schools. 
Thus, any lack of appropriateness of the content of the tests, or the 
testing procedures, should not contribute any bias to the evaluation of 
the impact of the offer of incentives. 

Test Content . Tables II-2 and II-3 present the skill content for 
Reading and Mathematics, respectively, of the five test batteries in the 
Metropolitan Achievement Test series. The subtest scores are added to- 
gether to produce a Total Reading or Total Mathematics score. Subtests 
that did not contribute to these scores were not administered. 

Test Reliabilities . The publisher's estimates of test reliability 
were derived using Saupe's estimate of the Kuder-Richardson Formula 20 
and are presented in Table II-4. Because the publisher's standardization 
population was so dissimilar to this project's population, reliability 
estimates computed by the Battelle Memorial Institute (Ray, et al.j 1972) 
have been included in Table II-4. The Battelle reliabilities were de- 
rived from the MAT 1970 used for the evaluation of the GEO performance 
contracting experiment conducted during the 1970-71 school year. The 
GEO experiment involved low-income, underachieving students in Grades 
1-3 and 7-9 with a higher than average minority representation. The 
Battelle reliability estimates were derived using a more conservative 
formula (KR-21) than the publisher's (KR-20) . It is encouraging to note 
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TABLE II-2 



MAT SKILL CONTENT—READING 



TEST 
LEVEL 


APPROP. 

FOR 
GRADES 


SUBTEST 


DESCRIPTION 


NO. 

ITEMS 


Primer 


K.6-1.5 




Letters - recognition of capital and lower 
cooc j.6tters . 


11 






Reading 


Words - marking words that Identify common 
pictures . 

Sentences - reading sentences that describe 
a picture. (A few Items for the more able 
pupllSi used In post-test only.) 


16 
5 


Primary I 


1.5-2.4 


Word 

Knowledge 


A sampling of primary level words pre- 
sented In the familiar word**plcture asso- 
ciation format. 


35 






Reading 


Thirteen sentences In picture-sentences 
format, a few riddles, and short para- 
graphs • 


42 


Primary 
II 


2.5-3.4 


Word 

Knowledge 


A contlntiatlon of the word-picture asso- 
ciation format and Introduction of short 
written sentences to be completed. 


40 






Reading 


A few sentences In the sentence-picture 
format. Reading selections ranging In 
length from a few sentences to fully 
developed paragraphs. 


44 


Elemen. 


3.5-4.9 


Word 

Knowledge 


A sampling of vocabulary covering syno- 
nyms, classifications. 


50 






Reading 


Easy to difficult paragraphs with ques- 
tions on main Ideas, drawing Inferences, 
getting literal meaning, use of vocabu- 
lary In context. 


45 


Inter- 
mediate 


5.0-6.9 


Word 

Knowledge 


Covers a variety of words from social 
studies, science, humanities, general 
experience. Also antonyms, synonyms, 
classification. 


50 






Reading 


Selection ranging from simple paragraphs 
to fully developed topics with several 
paragraphs. Questions call for Identifi- 
cation of main Ideas, literal meaning. In- 
ferred meaning, and special meaning of 
words as determined by context. 


45 
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TABLE II-3 
MAT SKILL CONTENT— MATHEMATICS 



TEST 
LEVEL 


APPROP • 

FOR 
GRADES 


SUBTEST 


CONTENT 


o m 


Primer 


K. 6-1.5 


Numbers 


Concepts - basic ideas about time» money, 
geometry, counting, sets. 


20 






Computation - addition and subtraction al- 
gorithms with one-digit numbers. 


14 


Primary I 


1.5-2.4 


Computa- 
tion 


Free response items with addition and sub- 
traction (no regrouping) . Horizontal and 
vertical notation covered. 


27 






Concepts 


Counting, reading and writing numerals, 
place value, and other basic concepts. 
Simple dictated problems involving addi- 
tion and subtraction. 


35 


Primary 
IL 


2.5-3*4 


Computa- 
tion 


A variety of addition and subtraction 
examples some involving regrouping. A few 
multiplication examples. 


33 






Concepts 


Measurement, place value, laws and proper- 
ties and other modern concepts. 


40 






Problem 
Solving 


Seventeen teacher-directed items and eigh- 
teen pupil-read items on everyday numerical 
application. 


35 


Elemen. 


3*5-4.9 


Computa- 
tion 


Addition, subtraction, multinll cation with 
and without regrouping. Introduction of 
division and fractions. 


40 






Concepts 


Modern coverage of laws and properties, 
sets, measurements. 


40 






Problem 
Solving 


A variety of problems involving arithmetic 
operations. Simple chart reading. Select- 
ing appropriate number sentences. 


35 


Inter- 
mediate 


5*0-6.9 


Computa- 
tion 


Operating with natural numbers, decimals, 
fractions, percents; simple equations and 
negatives at advanced level. 


40 






Concepts 


Factors, primes, inequalities, estimation, 
laws and properties, concepts of fractions 
and decimals, other modem topics. 


40 






Problem 
Solving 


Application of geometry and measurement, 
use of charts and number sentences, multi- 
ple step problems, plus broad coverage of 
arithmetic operations* 


35 
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TABLE II-4 



RELIABILITY COEFFICIENTS FOR THE 
METROPOLITAN ACHIEVEMENT TESTS 







TOTAL READING 


TOTAL ARITHMETIC 


TEST 
LEVEL 




Publisher^ 


2 

Battelle 


Publisher 


Battelle 




(KR20) 


(KR21) 


(KR20) 


(KR21) 


Primer 


F 


.93 





.96 







H 


.93 




.94 




Primary I 


G 


.96 


.98 


.96 


.97 




F 


.96 


.92 


.94 


.88 


Primary II 


G 


.96 


.96 


.95 


.94 




F 


.96 


.94 


.95 


.98 


Elementary 


G 


.96 




.96 






F 


.96 




.96 




Intermediate 


G 


.96 


.94 


.95 


.95 




F 


.96 


.93 


.96 


.93 



Sources: 1. Harcourt Brace Jovanovich, Inc., Report No. 10, June, 1971. 
2. Battelle Memorial Institute, March, 1972. (Battelle used 
the Stanford Early School Achievement Test and the Califor- 
nia Achievement Test, Level I, for the first grade. The 
MAT Elementary Test was not used by Battelle because Grades 
4 and 5 were not In the OEO experiment.) 



that the two reliability estimates are very similar, despite the differ- 
ences between the norm and the OEO populations. Hence, because the OEO 
and the OE and the incentives project populations were quite similar, it 
is reasonable to conclude that the publisher's reliability coefficients 
are applicable to this analysis. 
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Achievement Test Administration 

Objectives. There were three objectives In the design and Implementa- 
tion of the achievement test plan. These objectives were: 

o To reduce the error variance attributable to the students' 

taking test levels that are Inappropriate to their achievement 
levels 

o To reduce the error variance due to variations In testing pro- 
cedures and to test circumstances that are not conducive to 
maximum performance 

o To protect the Integrity of the payment plan and the evaluation. 
Procedures . 

Test Level Assignment. Each student In the project was assigned 
to a test level that was appropriate to his or her current Reading achieve- 
ment evidence. Based upon a review of the schools' achievement prttems 
over the past years. It was decided that the test publisher's recommended 
level assignments were not appropriate to this population. Collectively, 
the students In this project were achieving at about one grade level 
below their grade placement. Therefore, It was felt that a test battery 
that was one level below what the publisher recommended would be more 
appropriate. At the same time. It was recognized that there was a con- 
siderable amount of variation within the grade levels and that the most 
appropriate procedure would be to administer several test levels within 
a grade level. Because this was a loglstlcally complex testing procedure, 
it was decided not to assign individually both Reading and Mathematics 
test levels. Table II-5 presents the Intercorrelatlons between the Total 
Reading and Total Mathematics scores for the MAT fall standardization. 
Following a review of these Intercorrelatlons and discussions with the 
test publisher, it was decided that the student's Reading level would be 
adequate for estimating the Mathematics test level. 
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TABLE II-5 

INTERCOmunONS, TOTAL READING AND TOTAL FATH, 
MAT, BY TEST LEVEL 



PRIMER 


PRIMARY I 


PRIMARY II 


ELEMEN. 


INTERMEDIATE 


.66 


.68 


.74 


.79 


.76 



Source: Harcourt Brace Jovanovich, Inc., Report No. 11, June 1971. 



To assign a pretest level to a student, his or her prior Reading 
achievement test score was used where It was available from school records. 
Only test scores that had been derived from a spring 1971 administration 
were used, so that In no site were the data more thai! four months old. 
For those students for whom there were no test data, the teachers esti- 
mated In a grade equivalent format, e.g., 1.4, 5.3, etc.j the students 
Reading achievement level as of September 1971. After the teachers did 
this, the class lists were returned to TAG and a test level assignment 
was then made. No students were assigned a test level higher than the 
publisher's recommendations, and In only a few cases were test levels 
assigned more than two grade levels below the recommended grade level. 

The same test level was used for the post- testing. It was recognized 
that this procedure might mean that some students' scores would be Inac- 
curate because the scale score would not be high enough to reflect their 
true growth. However, this limiting factor applied to less than 5% of 
the students, evenly distributed between experimental and control schools. 
Furthermore, the possibility that Item-analysis research might be done 
on these test scores made a test level change undesirable, because the 
subtests are not directly comparable across test levels. For these 
reasons, the pretest and post- test levels remained the same. Table II-6 
* gives the reader an Indication of the distribution of test levels by 
grade level and building throughout the total project. As can be seen from 
the table, the test level assignments were generallj* one to two grade 
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levels below the publisher's recommended level, with the number of below 
grade level assignments increasing as the grade level increases. For 
example, the Primary II level, recommended by the publisher for administra- 
tion in Grades 2 and 3, was not given to any second grade students; where- 
as 30% of all the Primary II tests were given to fifth grade students. 
This is congruent with the pattern of past achievement results revealed 
when the schools were selected (in which the schools fell increasingly 
behind the national norm as the grade placement increased) . 



TABLE II-6 

INCENTIVES IN EDUCATION 
TEST LEVEL ASSIGNMENT BY GRADE AND BUILDING, ALL CITIES 



TEST LEVEL 


GRADE LEVEL 


& PUB.'s 
. RECOMM. 


1 

E C 


E 


2 

C 


E 


3 

C 


4 

E C 


E 


5 

C 


E 


6 

C 


Primer 
(K.6-1.5) 


364 310 


187 


189 


122 


58 












Primary I 
(1.5-2.4) 




141 


124 


123 


132 


135 149 


13 


1 






Primary II 
(2.5-3.4) 








88 


161 


140 131 


172 


136 


91 


87 


Elemen. 
(3.5-4.9) 








11 


1 


125 106 


176 


183 


158 


195 


Inter- 
mediate 
(5.0-6.9) 














21 


46 


128 


153 



Test Administration^ TAC had the responsibility for selecting, hiring, 
training, and supervising test administrators for both the experimental 
and control schools in the four sites. All testing was under the super- 
vision of a TAC staff professional. Local substitute teachers were hired 
and given one full day of training in achievement test administration, 
with the following two full days used to administer the achievement tests 
to all eligible students. (Students whom the school district had classified 
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as retarded for any reason were not a part of the project.) This all took 
place on a Wednesday' through a Friday at each site. The following Monday 
and Tuesday were devoted to administering makeup tests to students who 
had been absent during the regular test administration. These procedures 
were followed for both the pretesting and post-testing. 

Table II-7 puts the pre and post-testing into the perspective of 
the total school year at each site. The project test sequence was set up 
so that each school district received the pretest three weeks after school 
had opened. The three week delay was chosen because it was believed that 
the students needed some time to become accustomed to the work-discipline 
skills that are so necessary in taking these achievement tests. The 
post-tests were scheduled about three weeks before the end of school so 
that the students would not be restless from their anticipation of summer 
vacation and the end of school. The times of pre and post-testing were 
coincidental with the dates on which the publisher had normed the tests — 
October for the fall standardization and Hay for the spring standardiza- 
tion. 



TABLE II-7 

SCHOOL YEAR AND PRE AND POST ACHIEVEHENT TESTS 



SITE 


SCHOOL 
FIRST DAY 


PRETEST 
FIRST DAY 


POST- 
TEST 
LAST DAY 


SCHOOL 
LAST DAY 


NO. 
DAYS, 
SCHOOL 


NO. 

DAY, 
PRO J. 


Z 

PROJ. 
YR. 


San Antonio 


8/30/71 


9/23/71 


5/4/72 


5/31/72 


180 


130 


72Z 


Jacksonville 


9/7/71 


9/30, 71 


5/4/72 


6/8/72 


180 


140 


78Z 


Cincinnati 


9/8/71 


9/30/71 


5/12/72 


6/14/72 


180 


139 


77Z 


Oakland 
















Longfellow (E) 


9/13/71 


10/7/71 


5/19/72 


6/15/72 


180 


130 


72Z 


SanCa Fe (C) 


9/13/71 


10/14/71^ 


5/19/72 


6/15/72 


180 


125 


69Z 



Notes: 1. The original Oakland control school withdrew from the project 
just prior to pretestings which necessitated a one-week delay 
to locate another control school and assign test levels to 
students . 
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Two full days were set aside for both the pre and post-test adminis- 
trations. Because students in any given classroom would be taking dif- 
ferent test levels, the students had to be regrouped according to the 
level of the test they were taking. Within these groupings, the range of 
grade levels was generally not greater than two, so that the older students 
would not suffer a loss of self-esteem from taking tests with the younger 
students. Table II-8 displays the two-day test program, by test level 
and by subtest. The amount of time available for the limited testing was 
more than sufficient to ensure that the tests were given in a relaxed 
manner, that there were ample breaks between the subtests, and that the 
dictated tests at the lower levels could be accurately administered. The 
regular classroom teachers remained in the test rooms, while the Planar 
examiners administered the tests. The classroom teachers assisted in the 
logistics of the tests and in establishing an atmosphere that was con- 
ducive to tes^-taking. 

Results. Despite the logistical problems inherent in multilevel 
testing, the pre and post administrations were accomplished with no 
problems that were likely to be an important source of error variance. 
For the major evaluation comparisons, by sites and by building, the test 
administration circumstances were essentially the same within the test 
administrations and between the pre and post-tests. For the payment com- 
putations, conducted for the experimental schools only, there was only 
one exception. A first grade classroom in Jacksonville was not pretested 
because the children had not had an opportunity to attend preschool and 
consequently had not learned how to hold a pencil. 

Obviously, any testing program that is sponsored by the Federal 
Government and upon which payment and professional self-image are contin- 
gent is going to be accompanied by some tension within the schools. 
However, the school faculties were highly cooperative and the achieve- 
ment testing conditions were more than adequate. 



11-16 



TABLE II-8 



TEST /UMINISTRATION SCHEDULE 



TEST 
LEVEL 


PROJECT 
GRADES 


TEST TIMES 


. MINUTES 


First Day 


Second Day 


Primer 

Reading 
Numbers 


1-3 


20 


25 


Primary I 

Word Knowledge 

Readine 

Math 


2-5 


15 


30 


Primary II 

Word Knowledge 
Reading 
Math Comp. 
Math Con. 
Probi. Solv. 


3-6 


18 
30 
18 


20 
25 


Elementary 

Word Knowledge 
Reading 
Math Comp. 
Math Con. 
Probl. Solv. 


3-6 


15 
25 
35 


25 

i30 


Intermediate 

Word Knowledge 
Reading 
Hath Comp. 
Math Con. 
Probl. Solv. 


5-6 


15 

25 
35 


25 
25 
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Attitude and Behavior Measurement 

Objectives 

Self-reports of attitudes and behaviors, observations of behavior, 
and pre-existing school building records of behavior were utilized in this 
project to cover as many outcomes as could reasonably be anticipated • In 
addition to the interest in student outcomes, teachers and parents were 
also included in the identification and development of outcome variables* 
Finally, in the original project design, the measurement of attitudes and 
behavior was to be conducted three times during the year in both the 
experimental and control schools. This was planned so that trends could 
be established which might provide some evidence of rates of impact over 
time and the sequences of change among the several outcome variables* 

Instrument Development 

-The process outlined in the introduction to this chapter was followed 
in the development of attitudinal and behavioral measures. At each of 
the four sites, all instruments developed for this project were reviewed 
and critiqued by school officials. In no case was any original outcome 
variable or item dropped from the instruments, although several changes 
in format or method of administration were incorporated based upon the 
school officials* recommendations. The general considerations underlying 
each of the instruments follow. Appendix A contains examples of the instru- 
ments developed by TAC for the project. 

To clarify questions that will undoubtedly arise from the discussion of 
and administration of various instruments, it is necessary to point out 
that the original design for the assessment of attitudinal and behavioral 
outcomes was not followed because of delays encountered in the OE and 0MB 
forms-clearance process. The teacher, student and parent questionnaires 
were submitted by TAC in October, 1971, and did not receive final 
approval until the end of February, 1972. The changes that resulted from 
this five-month review are worthy of note: 
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o There were 119 changes to the substance or procedures for the 
student questionnaire* 

o The parent questionnaire response scale was changed from a 

five- to a three-point scale because OE felt that a five-position 
scale was too complex for the project population. Follow-up 
procedures were modified slightly* 

o There were no changes to the substance or procedures for the 
teacher questionnaire. 

The original design of the project called for pre, interim, and post 
administrations of the instruments. Because of the forms-clearance 
delays, the questionnaires were administered only twice, in March and in 
May, an interval that hardly qualifies as a pre and post-test design. 

Student Q uestionnaire . The student questionnaire was designed to 
measure a student's attitudes toward the teacher, the school, himself, 
and his peers. In addition, it attempted to measure self-esteem, feelings 
of control over immediate and remote situations, and future expectations 
and aspirations. There were 43 items in the instrument and there was no 
fixed time limit for the group administration. Students in Grades 1-3 
were administered a practice questionnaire the day before the actual 
administration. For these grades, the TAC examiner dictated the questions 
to the students. The pages of the questionnaire were color coded with 
one question per page; In Grades 4-6, the questionnaire was self-paced. 
Twelve items were in the smiling-face, neutral-face and sad-face response 
format, with the responses for each question in the same order of presenta- 
tion. The remaining 31 questions were in the balloon-child and flag-child 
response format. For each item, the balloon child appeared on the left 
and the flag child on the right. However, in some cases, the balloon 
child was associated with the more desirable answer choice, and in other 
cases with the less desirable answer choice. This was done as a means 
of eliminating some of the response halo effects that might otherwise 
operate. In both types of questions, the student was asked to choose the 
face or figure most like himself/ 
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All students in the project were twice administered the questionnaire 
in the classrooms. The first administration was delayed by the required 
forms-clearance procedures from October, 1971 to March, 1972. The second 
administration was in May, 1972. 

Student Interview . The student interview contained 15 items, five 
of which were self-reports on behavior and ten of which were assessment 
of the student's attitudes. The interviews were' conducted by a TAG field 
representative in the school buildings and the student's responses were 
coded by the field representative during the interview on a five-point 
scale, ranging from positive to negative. The first part of the question 
was designed to give the. student an opportunity to elaborate on his 
feelings toward the topic under consideration. This elaboration was 
noted by the field representative; and if a conflict appeared between the 
student's discussion and his subsequent response, the field representa- 
tive probed further to ensure that the student's attitudes were accurately 
recorded. The questions covered attitudes and behaviors toward school 
in general, teachers, peers, reading and arithmetic, and future expecta- 
tions. 

Between 40 and 50 students per site (20-25 per school building) were 
interviewed twice during the year. The first interviews were conducted 
during the months of November and December, 1971, and the second interviews 
were conducted in April, 1972. Unless the student had left the school 
district, the same students were interviewed both times. Students were 
randomly selected by building, grade level, and classroom. 

Parent Questionnaire . The parent questionnaire contained 45 items. 
Part I of the instrument contained 18 items designed to measure on a 
three-point scale attitudes toward their child's school, teacher, and 
academic progress. Twenty items were self-reports of the parentis or 
child's behaviors in areas thought to be relevant to academic perform- 
ance. Part II of the instrument contained seven questions about the 
family's demographic characteristics. Parts I and II were administered- 
in March, 1972; and Part I only was administered in May, 1972. 
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All parents in the project received the questionnaire which was dis*- 
tributed at the end of the school day and taken home by the students. The 
parents were asked to complete the questionnaire and mail it directly to 
TAG in a pres tamped and addressed envelope. Three weeks after the question- 
naires had been distributed, the nonrespondents were randomly sampled by 
site, building and grade level. No less tlian 50 parents per building were 
selected for the follow-up and were directly mailed a second questionnaire* 
Ten parents per building were sampled from those who did not return the 
follow-up questionnaire. They were telephoned at their homes by the TAG 
field representatives and encouraged to return the questionnaire. 

Parent Interview . The parent interview contained 20 items, 13 of 
which were self-reports on behavior and seven of which were attitude 
assessments. The interviews were conducted in the parentis homes, and 
all responses were recorded by the TAG field interviewer. As with the 
student interview, there were two parts to each question, the first part 
open-ended and the second part closed and direct. The response to the 
second part was recorded. The questions on attitudes were recorded on 
a five-point scale ranging from positive to negative. The questions on 
behaviors were also recorded on a five-point scale with frequency counts 
assigned to each interval to eliminate subjectivity in recording the 
response. 

Between 40 and 50 parents per site were interviewed in November, 
1971, and in April, 1972. They were the parents of the students who had 
been selected for the student interviews, and they were interviewed in 
both the fall and spring. 

Teacher Questionnaire . The teacher questionnaire contained 39 items. 
Part I of the instrument consisted of 25 questions about the teacher's 
attitudes toward the students, the parents, the profession, and the 
project; and three questions about teaching behavior. Part II contained 
11 demographic and background questions. Parts I and II were administered 
to all teachers in March, 1972; and Part I only was administered in June. 
1972. 
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The questionnaires were distributed in the schools, and the teachers 
returned them directly to TAG In a prestamped and addressed envelope. 
Nonrespondents were not follcrwed up. 

Teacher Interview. The teacher Interview contained 14 Items, seven 
of which measured attitude and seven of which measured behavior. All 
Interviews were conducted In the schools, and responses were recorded by 
the TAG Interviewer during the Interview. Again, like the student and 
parent Interviews, there were two parts to each question, one part 
open-ended and the other part closed. The open-ended responses were used 
by the Interviewer to "break the Ice" with the teacher and also to 
determine the accuracy of the categorization of the cloded response 
question. The responses to the attitude questions were recorded on a 
five-point scale ranging from positive to negative. The behavior questions 
were also recorded on a five-point scale with frequency counts assigned 
to each Interval on the scale. All teachers were Interviewed, once In 
November, 1971, and a second time In May, 1972. 

Glassroom Observation. The classroom observation Instruments were 
designed for the systematic and direct categorizing and recording of. 
student and teacher behavior. The Instrument had 14 types of teacher 
behavior, e.g.j testing, lecturing, praising, etc.; and four context 
categories—one student, small group, large group and total class. There 
were also 14 categories of student behavior, e.g.^ reading, listening, 
criticizing, etc.; and four context categories—alone, small group, large 
group and total class. 

Glassroom observation data were collected on every project teacher. 
There were two rounds of classroom observations, the first round In 
November and December, 1971, and the second round In April, 1972. During 
each round, a teacher's classroom was observed four separate times, about 
15 to 20 minutes per observation. The TAG field representative's sequence 
of observations was as follows: 

o Allow time for the class to settle down and for the lesson to 
begin. 



11-22 



o Begin the observations with the teacher* Observe, categorize, 
and record the behavior every ten seconds for three minutes* 

o Then observe the students In the same manner for the next three- 
minute cycle. Each ten-second observation was taken on a 
different student* 

o Go back to the teacher for another three-minute cycle and then 
to the students for a final three-minute cycle* 

The four observations were added together each round to give pre and 
post observation scores* 

School Records * Usually, a sad tale can be told whenever an attempt 
Is made to use school document files for evaluation purposes; this project 
Is no exception* Unproductive but substantial efforts were made In the 
following areas: 

1* Attendance * The original plan called for collecting attendance 
data by Individual student* This was found to be too time con- 
suming; consequently, attendance data were aggregated and 
collected by homeroom group* 

2. Tardiness * All schools in the project were supposed to keep 
tardiness data* In fact, only about 60% of the teachers kept 
reliable records of this behavior and there was no uniformity 
even within buildings* This effort was abandoned* 

3* Library Use * Although all schools had libraries, some schools, 
because of the nature of their operation, were unable to keep 
any useful information on library use, and others could not even 
operate the library because of reductions in the budget* This 
effort was abandoned* 

4* Other Information * In those districts that used th*.* MAT in 

their achievement test program, the possibility of using post- 
test scores for a trend analysis was explored* This turned out 
to be unfeasible because mobility from year to year reduced 
the number of students to an unworkable sample* 
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Other Information (continued) . 

The use of school grades was explored, but different grading 
standards among the districts and within schools in one district, 
(Grades 1-3 used satisfactory-unsacisfactory whereas Grades 4-6 
used A through F) precluded the use of this variable ♦ 

Disciplinary referrals were examined, but it was apparent 
that neither the referring teachers nor the principals' offices 
in any building maintained these data* 

In summary, school building records were a disappointing 
source of reliable information* If the project had been planned 
to extend over several years, an information system might have 
been established that would have been as xiseful to the adminis- 
tration of the school as to the evaluation of the project. 
Obviously, the benefits of trend analyses are far greater than 
a one-shot experimental design; and the combination of a decent 
design and execution with historical information would have pro- 
vided rich and reliable data not commonly found in educational 
evaluations. It should be noted, however, that the schools are 
really not at fault for these data management shortcomings. 
Schools do maintain reliable data when those data are useful to 
them. For example, attendance and enrollment data that generally 
determine State funding allocations were maintained and collected 
in all buildings in the project. The failure of the schools to 
keep information beyond mere attendance data is an outgrowth 
of the kinds of questions that are asked and the incentives 
offered for maintaining the information. A one-year, one-shot 
demonstration project is not a sufficient incentive to the 
schools to rearrange their normal recordkeeping procedures. 
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CHAPTER III 
EVALUATION 

Evaluation Methodology 

EvalTiatlon Strategy 

The strategy that has been adopted for this evaluation analysis has 
several elements. Each of these elements was chosen in light of the policy 
objectives of the project, the constraints that actually operated in the 
field settings, and the need for rapid processing of the data and delivery 
of a final ev.^Iuation report. 

The strategy included the collection of a considerable variety of data 
achievement test scores i questionnaires, interviews, systematic observation 
and analysis of school records—from the students, the teachers, and the 
parents. Much of this data was collected at more than one point in time. 
The purpose in collecting this widely diverse data was to provide maximum 
possible opportunity for anjr impacts of the incentives program, not just 
student academic achievement, to manifest themselves and be noted. The 
complementary efforts of MAC and TAC along these lines, in which MAC 
carried on formal observations and interviews at each site, and TAC 
collected the more quantitative data, were quite extensive. An effort was 
made to **leave no stone unturned** in detecting the potential consequences 
of the incentives program, and identifying the conditions which facilitate 
or inhibit its impact. 

A second feature of the evaluation strategy was an effort to organize 
the research questions specifically around the policy issues described in 
documents provided by OE. To this end, a large number of hypotheses were 
formulated, involving considerable detail. These hypotheses then were 
organized according to a classification scheme that allowed them to be 
considered in groups. The classification scheme has three main components: 
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1. The particular comparisons that are to be made (e.^., Parent- 
Teacher Model versus Teacher-Only Model) 

2. The persons to whom the hypotheses refer (,e.g.t students) 

3. The specific dependent variable being examined {e.g., rea<?-'ng 
achievement) . 

In general, all hypotheses were formulated so that the more favorable out- 
come was predicted for the treated group, and for the Parent-Teacher model 
(hereafter PT) in comparison with the Teacher-Only model (hereafter TO). 
In this introductory section, the first of the classification components— 
the particular comparisons to be made— will be discussed in some detail. 
Discussion of the other aspects of the hypotheses classification will be 
deferred until the sections on results. It should be noted here, however, 
that the number of specific hypotheses being examined is quite large. 
Thus, unless some organizing scheme and some summary hypotheses are used, 
there can be a real problem of losing sight of the forest amid the trees. 

A third feature of the evaluation strategy was the decision to concen- 
trate primary attention on the practical importance of whatever impacts 
night be found. In other words, the report will not merely report the 
statistical significance of differences in the outcome variables. In all 
too much educational research, including some very recent work, the 
investigators report that the treatment produced a statistically signifi- 
cant result in favor of their hypotheses, but give scant attention to the 
more important point that the actual size of the difference found is so 
small as to make it negligible from a practical point of view. Thus, 
the strategy adopted here was that statistical significance tests should 
be used primarily for one purpose: to verify that a difference which 
seems to be practically important (is numerically large in comparison to 
what was hypothesized), is not due merely to chance. Of course, in any 
analysis where a large number of comparisons are being carried out, some 
of these comparisons will exceed the critical value of the statistical 
test simply by chance. This is unavoidable. The appropriate response to 
it is to use caution in interpreting individual statistically "significant" 
results, especially if they are not part of a substantively coherent pattern . 
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The statistics, used in tests of significance do have a second kind of 
application, however; and that application will be employed in this report. 
Such statistics as, for example, the value of student *s t, or the point 
biserial correlation coefficient from a least squares fit, can serve as 
summary descriptions in common and familiar t^rms of the size of a parti-- 
cular difference relative to some other difference. Some of these summary 
statistics will be used as a means of presenting the results of comparisons. 
Substantive patterns in the- results will also be identified as an addi- 
tional means of deciding whether the separate results reflect some real 
and dependable impact of the incentives models. 

A fourth element in the evaluation strategy was made possible by the 
design of this project, and made desirable by the nature of the audience 
for this report. That element is an attempt to rely wherever possible on 
simple and direct statistical methods. This will make the meaning of the 
analysis clearer to many readers. It will also minimize the number of 
debatable assumptions that are required for the application of the analy- 
tical method. It is a general principle that the more elaborate and 
sophisticated the statistical method, the more complex and questionable 
are the assumptions required for its legitimate application. In this 
report, statistics which are direct and straightforward will be applied 
wherever possible. The possible slight loss in "efficiency" incurred by 
this decision is not a problem, because it will not bias the estimates 
of the size of the treatment impact. At the worst, these simpler statis- 
tics will have slightly less px^ecision of estimate, or will be slightly 
conservative as to the question of statistical significance. This is not 
a problem, given the policy-related purpose of this study. 

Analysis Decisions 

These general elements of the evaluation strategy prompted several 
decisions early in the evaluation analysis. Some of these are discussed 
here; others will be mentioned as they arise in the detailed presentation 
of results. The first decision concerned the score format to be used in 
analyzing the achievement test data. Because the analysis Involved only 
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internal comparisons among the students involved in the project , there was 
no actual need for using scores which are refer rable to national norms. 
However 9 it was thought desirable to relate the results of the achievement 
test analysis to the typical scores obtained by average students, so that 
the practical magnitude of the treatment impact can be judged more readily 
by the reader. 

Three score forms were seriously considered for use in the achieve- 
ment test analysis. These were: raw score, grade equivalent score, and 
tect publisher's standard score. Note that the "standard score" does not 
refer to a score obtained by subtracting the sample mean and dividing by 
the sample standard deviation (as the term is often used in statistics). 
The main reason for not using the raw score was that the multilevel test 
administration, (in which each child in a class was tested with the test 
level deemed most appropriate to his personal achievement level at the 
time of testing) made any summary of the raw scores not convarable within 
or among the four sites. If raw scores had been used, an equating for- 
mula developed on only the project data would have to be created. Such a 
development would have been possible in principle, but in practice it 
would have been difficult and undependable, given the relatively small 
size of the sample taking each test level. For these reasons, the raw 
score analysis was rejected. 

This left the standard score format and the grade equivalent format 
as possibilities. In the end, it was decided to adopt the standard score 
format, for reasons which will be-^explained below. First, however, it 
is necessary to describe briefly the standard scores and their relation- 
ship to grade equivalents. The standard scores are the score form used 
by the test publishers as the basic benchmark from which the grade equiva- 
lents are calculated. That is, the raw scores on each form and level were 
calibrated by the equipercentile procedure, and the resulting common 
scale, after some further smoothing, were used as the test publisher's 
standard scores (Orr, i972). Thus, the standard scores are, in a certain 
sense, more fundamental than the grade equivalents because there is one 
less smoothing operation involved in their calculation. 
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Another argument leading to a preference for the standard scores is 
that they have less of what Donald Campbell (Educational Testing Service » 
Inc. 9 1970) has called "fan spread". That is, the standard deviations 
of standard scores for different test levels are more nearly constant 
than the standard deviations of the grade equivalents. Table III-l shows 
the standard deviations for the norm groups on several of the test levels 
at several grade levels. As can be seen from the table » there is con- 
siderably more spread between the standard deviations in the grade equiva- 
lent scores (a range from 0.61 to 2^07 for Total Reading » and from 0.71 
to 1.77 for Total Hath) than there is for the standard scores (a range of 
19.0 to 14.7 for Total Reading and from 10.4 to 12.6 for Total Math). 



TABLE III-l 

STANDARD DEVIAIIONS OF METROPOLIXAK ACBXEVEHENT TESTS 
DERIVED SCORES FOR PUBLISHER'S NORM GROUP 



TEST 
BATTERY 


GRADE 
LEVEL 
AT WHICH^ 
AnONIS . i 


STANDARD DEVIATIONS 


TOT READ 


TOT MATH 


G.E. 


STAN 


G.E. 


STAN 


Primary I 


1.7 


0.61 


10.0 


0.71 


12.6 


Prinary I 


2.1 


0.77 


10.9 


0.74 


12.1 


Primary 11 


2.7 


1.01 


10.9 


0.86 


11.1 


Primary II 


3,1 


1.17 


11.6 


0.92 


li.4 


Elementary 


3.7 


1.38 


13.0 


l.U 


12.0 


Elementary 


4.1 


1.61 


14.0 


1.19 


12.0 


Elementary 


4.7 


1.67 


14.3 


1.35 


12.1 


Intermediate 


5.1 


1.83 


13.5 


1.29 


10.4 


Interaediatl 


5.7 


1.88 


13.0 


1.51 


12.2 


Intfermeifiate 


6.1 1 


2.07 


14.7 


1.68 


12.1 


Intermediate 


6.7 


1.97 


13.5 


1.77 


12.7 



Source.:' Harcourt Brace Jovanovicht Inc.^ Report No. 8, June 1971. 
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In short, although either of these score forms will be affected by the 
fan-spread phenomenon, that effect will be less severe for the standard 
scores than for the grade equivalent scores* 

An additional advantage of the standard scores in that the procedure 
used to create them and link the different levels of the test serves to 
justify the claim that the standard scores reflect equal intervals on the 
underlying achievement dimension being measured. In other words, the 
standard scores can justifiably be treated as interval scores. This is, 
in fact, their defining characteristic, and the one irfiich makes it legiti- 
mate to carry out averaging and other arithmetic operations on them. 

Also, the standard scores are more precise than the grade equivalents. 
Between the grade equivalent scores of 1.1 and 6.8, there are 57 units 
on the grade equivalent scale (as far as the actual data coded on the tape 
supplied by the scoring service is concerned). For the Total Reading 
Subtest, however, there are 70.5 standard score units («85*5 - 15.0) 
between these same two points. Thus, the standard scores divide up this 
interval more finely than the grade equivalent scores do. Similarly, for 
the Total Mathematics Subtest, there are 71 («95.0 - 24.0) standard score 
units in the interval between 1.1 and 6.8 on the grade equivalent scale. 

Finally, as a purely practical matter, the standard scores are the 
only form of derived scores available for the Primer Level Test. No grade 
equivalents have been defined for this test by the publisher. 

There is one disadvantage to the standard scores, however. Standing 
alone, they do not provide the reader with an intuitive appreciation of the 
magnitude of the gain made by the students. From the point of view of 
the researcher this is an advantage, because it focuses attention on the 
relevant comparison between experimental and control, rather than on the 
less relevant comparison between these students and the national norm. 
However, it does nothing to assist educators in determining the practical 
importance of the findings. 

It w ould be desirable if some benchmark were available to indicate 
the amount of growth to be expected for the test publisher's norm group. 
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Hovever, this cannot be done with precision* Because there is little avail- 
able data on "pre" and "post" standard scores on Metropolitan Achievement 
Tests on large 'samples of populations such as the one involved in this 
project 9 no direct comparls ns vith a familiar reference group can be 
made* Also^ because the growth pattern in the standard scores is nonlinear 
(being a negatively accelerated curve similar to those published by 
Science Research Associates (1967)), there is no direct conversion possi- 
ble between average growth and the publisher's norms for individuals* In 
an effort to develop conversion formulas so that a familiar benchmark 
could be used, several different calculating procedures were created; they 
each yield sofiei^t different results* However, as an approximation, it 
can be stated that for the publisher's norm ^ group, a school average gain 
of approximately 9 to 11 standard score units would be expected in the 
time period covered by the project* 

With, these benchmarks in mind, one can use the standard scores for 
the analysis, and yet refer to a familiar reference point for determining 
the educational significance of differences which are found* In the 
following pages, reference will be made to this benchmark whenever the 
occasion warrants* 

Xn addition, to .further specify the practical magnitude of treatment 
Impact in educational terms, we have calculated the percentage by which 
the observed gain in standard scores for the esqperimental group exceeds 
the observed gain for the control group; This is determined by dividing 
the difference between the experimental group gain and the control group 
gain by the control group gain* For example, if the experimental group 
gain were- 7*5, and the control group gain were 5*0, then the calculation 
would be: 

7.5-5.0,2^, 
5.0 5.0 

Although the exact limits are somewhat arbitrary, for purposes of inter- 
preting the educational significance of the marginal gains, the follc-vlng 
categories were established: 
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o 0 to 14.9Z net gain is negligible, 

o 15 to 29 .92 net gain is small, 

o 30 to 49 .92 net gain is medium, 

o 50Z and more net gain is large. 

Thus, in the example just presented, the EXP school net gain of 50% would 
have a "large*' educational significance. This line of reasoning seems con- 
sistent also with the fact that the rate of gain for students in control 
schools of this sample is approximately 65-70Z of the rate of gain for the 
publisher's norm group. Thus, a large effect in our terms would be one 
which brought the galtt rate of the target population to be nearly equal 
to that of the publisher's norm group, and this would be almost exactly 
a 5QZ increase in gain rate. 

A second matter that has to be decided is the exact nature of the 
comparison statistics to be used. For the present report, the^ primary 
mode of comparison adopted was the student t-test. This test is carried 
out by pooling the data from all students (or teachers or patents) who 
are properly Included in one side of the comparison, finding their average 
and standard deviation, and then doing the same for all data properly 
Included in the other side of the comparison. Then, these two means are 
cooq^red, with their standard deviations being assumed equal in the popu- 
lation, and a t-tcst is calculated. The resulting t-test is then pre- 
sented along with the means, standard deviations, and number of cases, in 
a summary table. 

A technical feature inherent in this procedure is that in some in- 
stances it may not be the most efficient test possible. That is, it may 
leave as part of the error variance (via the standard deviations used in 
the t-test) some components which could have been extracted. Although this 
fact is recognized, it is not regarded as a weakness. The reasoning is 
as follows. First, for most of the comparisons of direct interest, and 
particularly for those in which it might be possible to extract variance 
due to other components (e.g.^ gjade, site), the number of cases in the 
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comparison is so large that there is no real likelihood o£ missing an 
educationally sizable effect because of the limitations of the significance 
test. On the contrary, several of the results turn out to be statistically 
significant, but so small in practical terms that there is no point in pur- 
suing them any further. Thus, the emphasis in this project is not on 
extreme precision of statistical testing, but for good estimates of the 
size of educational effects and for understandable procedures. 

The t-test used here is a statistical technique that is usually 
covered in first courses in statistics and so is familiar to most persons 
who will be reading this report. Its application here is an entirely 
justifiable one because all of its assumptions are easily met (at least 
in the analysis of the achievement test data). The distributions expected 
from these tests are ^rnnctrical and approximately normal, the variances 
are almost certainly eq^al in the population, and the samples being com* 
pared are always statistically independent and not too different in a 
number of cases. 

In addition to the direct analysis of differential gains, a supple- 
mentary analysis was also performed on the achievement test data. This 
supplementary analysis involves again the use of the t-test, but the test 
was applied separately to the pretest data and to the post-test data. - 
This supplementary analysis is presented in the section f olloi^ng the main 
analysis. 

Organization of Findings 

In terms of the broader org^ization of the hypotheses which was 
mentioned earlier, all of the material in the main and ^Supplementary 
analyses of this section deals with achievement test data as a variable 
and with students as the persons being exacined. These analyses are 
oriented to a set of comparisons developed out of the combined work of 
OE, TAG, and MAC personnel. They represent a distillation of ideas 
arising from direct policy interest and those arising as a result of sub- 
stantive questions about: the possibility of "site-by-treatment" inter- 
actions. They are summarized la Table III-2 and described on the following 
page. 
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TABLE III-2 

LIST OF COMPARISONS MADE TO EXAMINE PROJECT RESULTS 

Single-Site Comparisons 

^" Experimental school v ersus Control school. Cincinnati, all grades 
coiBbined. 

^* Experimental school versus Control school. Jacksonville , all grades 
combined. 

^' Experimental school versus Cont rol school. Oakland, all grades com- 
bined. 

Experimental school versus Control school. San Antonio , all grades 
combined. 



Model-Based Comparisons 

5. Experimental school versus Control school. Teacher-Onlv Model 
(Jacksonville and Cincinnati) . 

6» Experimental school versus Control school. Parent-Teacher Model . 
(Oakland and San Antonio). 

^* Parent-Teacher Exper imental schools versus Teacher-On ly F.vppr^1lu.n^a^ 
schools (Oakland and San Antonio/ Jacksonville and Cincinnati) . 

Parent-Teacher Expe rimental and Control schools versus Teacher-Oniv 
Experimental and Control schools " 

8* Experimental schools, all cities, versus Control schools 4 all cities . 

Comparisons 1, 2, 3, and 4 in the above table involve pooling data 
from all the grades at each school at each individual site and comparing 
the results of the experimental school at a given site with those of 
the control school at the same site. These four comparisons represent 
the mo£t direct and focused test of the hsrpotheses about the impact of 
the two incentives models that can be made. It should be raneobered that 
the models were appUed to the schools as intact units and that in the 
design of the project there was no policy interest in differential effects 
of a given model at different grades. If such differential effects do 
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exist, unless there is a consistent pattern to these effects, (such as 
greater gains in the lower grade levels across sites) , they are irrelevant 
interactions. To examine these questions, separate tabulations have been 
run for each grade on the achievement test data. The findings of these 
grade-by-grade analyses are discussed later in this report. However, 
these grade-by-grade results are of only secondary interest. The compari- 
sons of what is called the "EXP" (for experimental) and "CON" (for control) 
entire schools at each site convey the principal results of the project. 
Thus, the single-site comparisons make up the first four of the summarizing 
comparisons. 

Comparisons 5 and 6 aggregate the data by the type of model used and 
compare conbined EXP against combined CON. Comparison 5 pools the data 
from the two sites at which the Teacher-Only model was used (Cincinnati 
and Jacksonville) and compares the results in those two EXP schools with 
the results in the two corresponding CON schools. Comparison 6 involved 
a similar comparison of the Parent-Teacher model at two sites (Oakland 
and San Antonio). These comparisons offer a good example of why statistical 
elegance is not needed here, and would be distracting if used. The main 
question, as already indicated, concerned the effect of the treatment 
models. There are two kinds of treatments, the PT model and the TO model. 
To assess the overall effect oF. the TO model, one must examine the results 
in the two sites in which it was applied. If those results are similar in 
both sites, then it can be concluded that there was no "site-byrtreatment" 
interaction for the TO model, and the two impacts can be averaged for a 
more precise estimate of the overall impact of the TO model. If there is 
an effect at each of the two separate sites, and that effect is statisti- 
cally significant as well as substantively nontrivial, then it is inevitable 
that the average of those two effects also will be statistically signifi- 
cant. 

On the other hand, if the sites have impacts that differ widely 
from each other, then an averaging of their effects will not be completely 
interpre table; hence, it will be less useful. This situation would arise 
because averaging suppresses an aspect of the phenomenon, namely site-to- 
site differences in responsiveness, that is of major policy interest. 
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Under the circumstances that there are site-to-site differences in respon- 
siveness to the incentives, the averaging together of two sites is at 
best a very crude indicator of a hypothetical average. The limitations of 
this pooled average under those circumstances must be strongly emphasised. 
The sites examined are only turp in number, and they are neither probabilisti- 
cally nor even representatively chosen. Thus, generalizations from them 
to some larger population of potential sites are almost completely judg- 
mental. Nonetheless, since this is a central focus of policy interest, 
the model based comparisons are provided* 

Comparison 7 was carried out to address the question of the differen- 
tial impact of the PT model in comparison with the TO model. Obviously, 
the PT model is an extension of the TO model, albeit a potentially expen- 
sive one because incentives are offered to parents as well as teachers. 
Therefore, from a policy point of view, this model should be seriously 
considered only if it produced substantial marginal benefits beyond those 
obtained from the TO model. As one way to address this question. 
Comparison 7 pooled the two EXP schools in the PT incentives model and 
separately pooled the two EXP schools in the TO model. It then compared 
the outcomes in the PT-EXP schools with the outcomes in the TO-EXP schools. 
In other words, the TO-EXP schools in this comparison were used as the 
"control" group for estimating the marginal impact of the PT incentives 
model. 

This is intuitively a legitimate comparison, because the PT incentives 
model includes all the features of the TO incentives model, plus others 
as well. Note, however, ^at the comparison just described does not 
eliminate site-to-site variation in learning rate as a source of error 
variance and possible bias. Stating this differently, the comparison does 
not answer t|ie question: Is the net impact of the PT incentives model 
substantially greater than the impact of the TO incentives model, when 
site-to-site differences in learning rate are taken into account? To deal 
with that question, a special comparison was performed on the achievement 
gains analysis (Comparison 7a in Table III-2); it is described later. 
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The last comparison Is an overall one* It suppresses all differences 
between the different incentives models, and pools across all four sites* 
This comparison is in many respects vague and imprecise, but it is the 
best that can be offered to answer the very broad question that policy 
interest raised: Do the incentives, considered overall, seem to have a 
sizable Impact? In one sense, this last comparison is oversimplified* 
Yet, it is included for the benefit of those who wish to see a very general 
summary comparison. Clearly, this overall comparison would have the most 
interpretability if the PT treatment showed no additional impact over the 
TO treatment, and if there were no site-by-treatment interactions* As 
will be seen in later sections, these conditions do not hold for the data 
at hand , and so the comparison is less meaningful * 
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I Achievement Data Analysis 

I Discussion 

The basic analysis of the achievement test data was carried out by 
I comparing the observed gains made by students in the experimental school 

(or schools) with the observed gains made by students in the corresponding 
control school (or schools) . These comparisons are the most direct and 
straightforward way to examine the impact of the incentives models. More- 
over, given the characteristics of the design employed in this project 
(i.e. J the use of intact, paired schools) these direct comparisons are 
methodologically the most appropriate technique available. 

Over the last decade, there has been much debate in the statistical 
I literature of the behavioral sciences as to the appropriate procedure for 



I dealing with change and growth data (Lord, 1956, 1958; Harris, 1963; 

I Bereiter, 1963; Lord, 1967; Coleman, 1968; Werts & Linn, 1970; Cronbach & 

I Furby, 1971). The problem which makes moot any analysis of change datri 

I is that for most kinds of data, the amount of change observed is dependent 



I (and predictable from) the initial level. As Coleman^s (1968) discus- 

I sion indicates, the substantive and methodological sources of that 

I dependence often are quite complex. Thus, the analytical question becomes: 

^ What is the most appropriate way to adjust for the impact of differences 

I initial level, so as to be able to ascertain the impact of other 

I influences on the amount of change? The untangling of all these components 

of the dependence of growth on initial level, particularly when errors of 
measurement as well as substantive forces must be taken into account, is 
difficult. Various lines of reasoning, and various technical approaches, 
have been suggested and are being explored, but at present it must be 
said that this vrhole subject area is one that is still in a state of 
development by statisticians and methodologists. 

' In the Incentives Project, however, the issues as to what adjustment 

j is appropriate are not very serious in practical terms. The design of 



I the study, and particularly the choice of schools within each site which 

I were as similar as possible with respect to average achievement level, 

i 

r 

i 
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make any adjustment largely a matter of refinement* The general direction 
of differences is not likely to be changed by the small adjustments which 
might be made* 

Although there are some differences between schools within a site as* 
far as initial score is concerned, these differences are largely due to 
random fluctuations, since the schools have been selected to be similar* 
These random fluctuations will tend to produce effects which make the 
treatment impact appear somewhat larger than it is* However, this is a 
known direction of bias, and so can be taken into consideration when the 
results are interpreted* For these reasons, it seemed that the simpler 
analysis without aiiy adjustment of scores — namely, a comparison of 
observed gains — was preferable to some adjustment that would introduce 
additional uncertainty. Thus, the direct comparison of observed gains 
has been used for the main analysis* 

However, because some distorting factors may be present, additional 
analysis^ techniques with the achievement test data have been employed 
in some further analysis. These other techniques isolate and adjust for 
certain of the distorting influences that may be present. The results 
obtained using these supplementary analysis techniques are reported in 
the next section of this chapter. In general, they .do not provide any 
grounds for substantially changing the conclusions drawn on the basis of 
the direct analysis of gains in achievement test score, although they 
do reduce the clarity of the results and the confidence that can be 
placed in them. In other words, the best guess as to what happened is 
the same, but its chance of being right shrinks somewhat. 

The results of the direct comparison of achievement test gains are 
summarized in Tables III"3 and III-4. Table III-3 summarizes the Total 
Reading results and Table III-4 summarizes the Total Mathematics results. 
The results shown in Tables III-3 and III-4 were computed using all students 
for whom Total Reading scores were available on both the fall (pre) and 
spring (post) test administrations. Each table shows the average gain in 
score, the standard deviation of the gains, and the number of cases in- 
volved for each of the comparisons previously discussed. 

i 
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TABLE III-3 
TOTAL READING GAIN 







GAINS 


STATISTICS 




COMPARISONS 


EXPERIMENTAL GROUP 


CONTROL GROUP 


t-val. 




% Net 






X Gain 


S.D. 


N 


X Gain 


S.D. 


,N 




Impact 
E/C 


1. 


E/C, Cin (TO) 


5.71 


5.29 


409 


6.39 


5.88 


362 


-1.69 


-.061 


-11% 


2. 


E/C, Jax (TO) 


6.90 


6.05 


738 


6.44 


5.59 


693 


+1.49 


+.040 


+ 7% 


3. 


E/C, Oak (PT) 


7.14 


6.11 


361 


5.04 


5.84 


487 


+5.09 


+.172 


+42% 


4. 


E/C, San (PT) 


8.07 


7.15 


333 


7.65 


6.66 


288 


+0.76 


+.030 


+ 6% 


5. 


E/C, TO 


6.48 


5.81 


1147 


6.42 


5.69 


1055 
775 


+0.22 


+.000 


+ 1% 


6. 


E/C, PT 


7.59 


6.64 


694 


6.01 


6.28 


+4.69 


+.122 


+ 26% 


7. 


PT-E/TO-E 


7.59 , 


6.64 


694 


6.48 


5.81 


1147 


+3.77 


+.088 


+ 17% 


7a. 


PT/TO 


6.89 


6.11 


1749 


6.29 


6.01 


1922 


+2.99 


+.049 


+ 10% 


8. 


E/C, All 


6.90 


6.16 


1S41 


6.25 


5.95 

1 


IdSO 


+3.25 


+.054 


+ 10% 



TABLE III-4 
TOTAL MATHEMATICS GAIN 







GAINS 


STATISTICS 




COMPARISONS 


EXPERIMENTAL GROUP 


CONTROL GROUP 


t-val. 




% Net 






X Gain 


S.D. 


N 


X Gain 


S.D. 


N 


'pb 


Impact 
E/C 


1. 


E/C, Ciu (TO) 


8.46 


8.03 


410 


9.57 


8.03 


358 


-1.92 


-.069 


-12% 


2. 


E/C, Jax (TO) 


9.90 


7.64 


734 


8.37 


7.37 


679 


+3.83 


+.101 


+ 18% 


3. 


E/C, Oak (PT) 


9.95 


7.68 


357 


6.73 


7.82 


488 


+5.96 


+ .201 


+ 48% 


4. 


E/C, San (PT) 


12.72 


9.96 


334 


8.84 


7.98 


287 


+5.30 


+ .208 


+ 44% 


5. 


E/C, TO 


9.38 


7.81 


1144 


8.79 


7.62 


1037 


+1.81 


+ .039 


+ 7% 


6. 


E/C, PT 


11.29 


8.96 


691 


7.51 


7.94 


775 


+8.56 


+ .218 


+ 50% 


7. 


PT-E/TO-E 


11.29 


8.96 


691 


9.38 


7.81 


1144 


+4.79 


+ .111 


+ 20% 


7a. 


PT/TO 


9.79 


8.27 


1728 


8.63 


7.91 


1919 


+4.32 


+ .071 


+ 13X 


8. 


E/C, All 


10.10 


8.31 


1835 


8.24 


7.78 


1812 


+6.98 


+ .115 


+ 23% 
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Each row of the table presents the results for one comparison pair 
of groups. Each row also gives the value of student *s t and the point 
biserial correlation coefficient (rpb) between treatment and gain* These 
point biserial coefficients seem small because they are based on students 
in all grades, so that any variation between grades is not taken out of 
consideration. For all the rows of the table, the group whose results 
are shown on the left is the experimental group (EICP) and the group on 
the right is the control group (CON). That is, the left side shows 
results for students in the school (or schools) receiving the incentive 
model (the presumably more powerful of the treatments) • 

Thus, the left side of the third row of Table 111-3 indicates that 
among all the children in the experimental school in Oakland, the average 
gain in Total Reading score from fall to spring was 7.14 points. Simi- 
larly, the right side of the third row indicates that for all the children 
in the control school in Oakland, the average gain in Total Reading score 
was 5.04 points. 

A formal test of significance logic has not been used, because the 
real policy question involves the magnitude of the differential gains 
rather than their mere existence^ However, the existence of nonaccidental 
differences in the gains is a necessary condition that must be met before 
it can be concluded that the treatment has an effect. For the purposes 
of an analysis of this sort, it is not misleading to use a t-value of 
approximately 2.00 as a criterion of statistical significance. 

As a caution, it should be noted that the existence of differences 
too large to attribute to sampling fluctuation does not, in itself, 
guarantee that these differences were caused by the treatment. The dif- 
ferences which are observed in any site are the net result of all the 
various systematic differences that exist between the two schools. The 
careful matching of schools in the design of this project has eliminated 
many of these systematic differences that might otherwise be present. 
However, it has not eliminated all such differences, and so there is no 
way to be sure that the outcomes are due to the treatment. Since only a 
small number of schools were involved, and since evidence in this data 
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indicates that there are substant-»al differences among these schools, there 
can be no appeal to ah argument that other differences will be averaged 
out over the many cases involved. In other words, the existence of a siz- 
able difference in the predicted direction (such as that observed for 
Oakland in Table III-3) is an encouraging sign, but cannot be taken as 
conclusive evidence of a treatment Impact. Conversely, the absence of 
such differences in outcomes may likewise be due to the operation of 
other differences besides those of treatment and control, and so negative 
conclusions also must be tentative. 

Table 111-3 presents the results for achievement gains in Total 
Reading score. This is referred to throughout the report as "Total Reading" 
or "Reading". Of the nine comparisons in this table, eight show differen- 
tial gains in the predicted direction (of course, several of these com- 
parisons are statistically dependent on the others) . When an unweighted 
analysis was done, to eliminate any distortion due to differences between 
schools in distribution of students across grades, seven of the results 
remain in the predicted direction. The exception was the E/C, TO cougar ison, 
row 5, which changed from +.22 to -.50. However, several of the differenr 
tial gains are quite small, and only five of the Total Reading t-values 
are greater than 2.00. Thus, the general though tentative conclusion is 
that the offer of ihcentives did not have a uniformly large effect on 
Total Reading gains. 

Single-Site Comparisons— Reading Gains 

In Cincinnati, the comparison of gains in Reading shows that students 
in the EXP school on the average gained less than the Cincinnati students 
in the CON school, but the difference in gain is negligible. Here, we 
must tentatively conclude that the incentives model had no beneficial 
effect on Reading achievement. 

The second row of Table III-3 shows the comparison for Jacksonville. 
As can be seen from the table, the differences in gains aire negligible 
although in the direction predicted. In this city as well, we must 
tentatively conclude that there is no beneficial effect of the incentives 
model on Reading achievement. 
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Students in the EXP school in Oakland gained more> on the average, 
than those in the COIJ school* Students in the EXP school, oh the a>^erage, 
gained 7 ♦ 14 points, and students in the CON school gained only 5.04 points* 
This is a difference in gain of 2*10 points, or 42% of the control school 
gain* A 42% net gain is at the upper end of the "medium" gain category 
presented on page III-7. Thus, the Total Reading impact in Oakland is 
clearly too large to be neglected, and it is nearly as large as might be 
expected if the incentives treatment were as beneficial as deserved • 

In San Antonio, the difference in gains is quite small, arid the t-value 
is less than 1*0. The direction of difference is, however, as predicted 
by the hypotheses. In San Antonio, there is no evidence of a beneficial 
impact of the incentives model on gains in Total Reading achievement. 

Note that, at this point, several complications begin to emerge from 
the pattern of results. These complications create difficulties over and 
above the other difficulties already mentioned. Generally, the incentives 
program seems to have had an erratic mpact oh Reading achievement. 
These variations in the: impact complicate the drawing of any concrete 
policy inferences. For instance, there is clearly a '^site" effect on 
Reading gains. The gains in San Antonio, for both the EXP and CON schools, 
are greater than the gains in Oakland.. A number of possible explanations 
for this can be offered. These differences in achievement gain may be 
due to differences ih the school character istica^^ or differences in the 
cities, or differences ih the region. Without considerable additional 
data, there is little that can be done to account for these differehces> 
although the process evaluation evidence (Educatioh Turnkey Systems, 
1972b, p. 65 ) sugg;ests that the administrative "climate" in San Antonio 
was more fayorabie than that in Oakland, and this may be at least a 
partial explanation. 

Clearly^ the variation in gain between untreated schools in different 
cities is approximately as large as the tapact of the treatment, and so^ 
the treatment cannot be universally recommehded, a not without 

much additional understanding of the conditio hs that cause the natural 
variations in fs^ih rate. 
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The site-to-s.ite differences in gain rate have other consequences 
as well. Because of the small number of sites; used, and the fact that 
not all three possible treatment levels were used in any site, (i.Si, 
the PT and TO treatment levels were never used in a single site together) 
there are problems introduced in drawing inferences about the relative 
effectiveness of the two incentives models. In particular, differences 
between the PT and the TO treatments are not easily distinguishable from 
differences betweoi sites in receptivity to the treatment. That is, the 
contrast between PT and TO treatments ;is_ confounded with the treatment- 
site interaction. 

The design used in the project cpatrolled for differences between 
sites in the composition of the student body, as far as initial score 
level is concerned, but it did- not control for differences in the current 
learning rate or receptivity among the four sites. If the sites where 
the PT treatment was used happened, to be operating with a higher general 
learning rate or were more receptive than those at which the TO treat^ 
ment was employed, then in Comparison 7 there; is likely to be an apparent 
positive effect of PT which is in reatlity due to differences in sites. 

Als6,/withiii the pair of sites that received the same incentives 
treatment there may also be differences between sites in their responsive- 
ness to the treatment. This seems- to be the case for Oakland and San 
Antonio with the Parent-Teacher incentive treatment. It appears that 
Reading scores in Oakland respond to the treatment rather substahtially, 
while San Antonio seems essentially unaffected. Because of this treatment 
site interaction, any broad: conclusions as to the effectiveness of a 
particular incentives model become quite tenuous. That is, the combining 
of Oakland and San Antonio experime^^^ schools into a single entity 
called the Parent-Teacher awdel population would lead one to draw conclu- 
sions only about what must be thought of as an imaginary average effect 
of' the Parent-Teacher model. 

There are two difficulties with the averaging concept. First, the 
average may be meaningful for policy making at the broadest Federal levels 
but it raises at least as many questions as it answers, because there is 
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no way to predict what the impact of the Parent-Teacher model will be for 
a given particular school i Second, the application of an average effect 
idea is weak here, because the set of units over which the average is 
calculated (two schools, one in Oakland and one in San Antonio) is small 
and unrepresentative of any politically meaningful group. 

Perhaps the most important inference that can b.e drawn from this 
discussion is that there are some very important factors— presumably ones 
operating at the school level, but perhaps also others, being determined 
by neighborhood or school system characteristics— that have been completely 
neglected in the design. This omission is not the result of negligence, 
but rather of ignorance. The field of urban education siaply 46es not 
offer firm knowledge about the particular variables, or combinations of 
variables, affecting tiie success of a school. Moreover, much of the 
available evidence indicates that many of the seemingly important factors 
are nonmanipulable by any fp kind of policy change. In short, 

the results here serve best: perhaps as an object lesson, in the pritditive 
state of knowledge about the factors contributing to school effectiveness. 
Because such knowledge is so limited, the other factors which affect 
eitiier tht ^t^noral learning rate of the responsiveness to an incentives 
treatmrat hava neither been held constant nor have they been measured 
so that they can be adjusted for iii the calculations; Yet, the data indi- 
cate that such influences are subs tant^ in size. 

Also, the possibility was examineci that the differential effects may 
arise out of a combination of the score format used (in which the typical 
gain is larger at the lower grades than at the upper grades) and the 
distribution of students across grades in the different schools, to 
check on this possibility, a separate analysis the achievement data 
was executed, using ah unweighted means analysis (Winer, 1962, p. 222-224). 
This analysis effectively makes the distribution of students across 
grades the same for every school, and so makes it impossible that dif- 
ferences in such distributions could be contributing to the effects ob- 
served, these results are not included here^ for lack of space, but they 
are very similar to the maiii analysis , and provide no reason to change 
any of the conclusions drawn. 
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Model-Based Comparisohs—Reading Gain 

The qualifications mentioned above are serious ones. From a metho-^ 
dolbgicaliy conservative point of view, they would be sufficient reason 
not to perform any further comparisons. Yet, these other model-based 
comparisons are not difficult to carry out, and may, if interpreted with 
the necessary caution, be of some use to policy makers and research plan-^ 
ners. For that reason, they have been calculated and are presented below. 

The first of the model-based comparisons was made by pooling the 
two sites which experienced the TO model. For these two sites, the direc- 
tion of effect differed, but the magnitude of the effect was negligible 
in both sites. When the two sites were pooled, these two opposite direc- 
tions tended to cancel one another, and the resulting average gain in the 
I EXP schools, 6.48, was nearly identical to that in the CON schools, 6.42. 
Thus, on balance, it .would seem that the TO model has no discernible 
impact oh Reading achievement in this data. 

The next modeirbased cbmpar constructed by pooling the two 

sites which experienced the PT treatment,, and comparing the EXP schools 
with the CON schools^. Here, the impact was in the predicted direction in 
both sitesi although it was much larger in one site (Oakland) than the 
other (San Antonio) i When the two sites were pooled, the average gain in 
the EXP schools was 7.59, and the average gain in the CON schools was 
6.6li This would seem to suggest, on balance, a non-negligible, but 
small, impact in the sites exposed to the PT treatment. The t-s tat is tic 
of +4.69 is consistent with this interpretation, and indicates that the 
difference almost certainly is hot due to chance statistical (fluctuations. 
Of course, as mentiohed earlier, such differences ^^m^^ due to 

differences between schools other than the treatment. In terms of impact, 
the net gain was small, the PT-EJOP schools gaihing 26% more than the PT- 
CON schools. 

The next comparison is a questionable one. It was aimed at contrast- 
ing the impact of the PT model with the impact of the TO model and made 
use only of the EH? students. It compared, on the one hand, students from 
EXP schools in Oakland and Sah Antonio with, on the other hand> students 
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from EXP schools in Cincinnati and Jacksonville • Thus, differences which 
are fpund may be due^ to differences in the two incentives mpdeisi to 
diicferences in the "main" effect of sites, or to ah interaction between 
the two factors. In short, it is a muddy comparisdh. However, it does 
show an apparent effect. That is, the average gain among the EXP students 
in the two sites which were exposed to the PT model is 7.59, and the 
average gain among students in the EXP schools in the two sites which 
were exposed to the TO model is smaller, only 6.48. Although the 
t-statistic was 3.77, the net impact was only 17%, which isi on the border^ 
line between "small" and "negligible". 

A more complicated comparison of the PT versus TO models can be con- | 
structed which eliminates the confounding between the site-to-site dif- 
ferences in growth and the net Impact of the PT treatment. This comparison 
(7a in Table III-2) includes taking the difference in gain between the 
EXiP and the CON: for the ^T sites, and comparing it with the difference 
in gain between the EXP and the CON for the TO sites . When this comparison 
is carried but, the difference in gain is no longer i. 11 (7,5? - 6.48), 
but inst'erd^is only 0,.^ 6.29) . This imquvi is still large enough 

to have a t-value of 2.?3. However, it is too small to be of any practical 
significance. 

The last cpmparisoK presented in Table 111-3 is ah overall comparison 
of the gains: in Reading score in all four experimental schools compared 
with the gains in all four control schools. It does not address any pre- 
cise hypothesis about the processes that are operating, but instead gives 
some general sense of the magnitude of the average impact that was: 
achieved. As can be seen, the difference in gains between the EJflP and 
the CON schools is not large (6.65) . 

Although the t-statistic was 3.25, this is an educationally negligible 
difference, for the net gain was only 10%. 
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single-site Comparisons--Mathematics Gains 

Table III-4 presents the results for achie^^^ gains in Total 
Mathematics score. This is referred to throughout the report as "Total 
^iath" or "Math"i It should be noted at this point that the Math test score 
gains, in almost ail instances ^ are larger than the Reading test gains. 
This is simply a function of the content of the test, and does not ihdir 
cate anything at all about the quality of instruction or relative perform- 
ance in the two subjects. There is no way to compare these scores across 
subject directly. Again, the results in each of the four sites are con- 
sidered individually i 1 

In Cihciimati, the comparison of the gains in Math indicates the 
students in the EXP school on the average do not gain quite as much as 
the students in the CON school i but the difference in gain is negligible 
(-12%) . The amount and direction of the Total Math results are similar 
to the results, obtained in Cincinnati for the Reading tests i 

For Jacksonviile , the average gain among the students in the E3ff 
school is 9.90 points, and the gain among stu^^^ in the CGN-schodl is 
8 i 37 points, yielding a difference of 1.53 points. This impact, although 
s ^t is tically large enough to be real (t-valiie of +3.83), produces an 
impact of only 18% which is just above the cutoff pbiht between the- 
"OTaU'' and '-negligible"^^^ g^ categories. 

The E3ff school in Oakland gained noticeably more, on the average, 
than the CON school in Oakland i The gains were, respectively ^ 9.95 and 
6.73. the difference is 3.22 points and is clearly statistically signi- 
ficant (t-yalue is 5,96) • the net impact on the EXP schobi is 48% and is 
oh the border between "medium" and "large", thus, the Total Math results 
in Oakland are similar to the total Reading results for Oakland. 

in San Antonio the E2ff students gain,, oh the average, 12.72 points ^ 
and the CON students gaihy only 8.84. This is a difference of 3.88 points 
and has a t-yalue of +5i30. The net gain is 44%, indicating a medium-to- 
large impact of the treatment (PT) in San Antonio. This is a clear 
instance of a substantial percentage impact of the treatment on the Total 
Math results, but hot on the Reading results, ih San ^tonio, 
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Model-Based C6mparisdns-r-Mathematics Gains 

Although Comparisons 5 through 8 in the table address hypotheses 
originally fori&ulated in the design of the project, they must be inter- 
preted very carefully in light of the^ patterns observed in the Total Math 
data* Here as in Reading, the gains in San Antonio, for both the EXP 
and the CON schools, are greater than the gains in Oakland. Also, there: 
are again some noticeable differences in response to the treatment 
between the two sites which are given the same treatment. For example, 
Cincinnati responds negligibly and in a negative direction to the TO 
treatment, but Jacksonville responds somewhat positively (though not 
enough to be important educationally). ^ 

As before, these patterns seriously restrict the generalizations that 
can be made. Yet, the model-based comparisons have been reported so 
that policy makers at all levels may have an understanding of the complexity 
inherent in interpreting the results . Much caution should be used in 
generalizing to policy interpretations from this set of findings. 

the comparison for the- two sites which experienced the TO model 
indicates a gain among the- EXP group 6> 9 .38 arid a gain, among the CON 
group of 8.78^ with the difference between these: being 0.60. Although 
this is statistically large enough ^to be corisidered, it_ is negligible 
educationally. Thus, the data indicate no impact of the TO treatment, 
on balance, keeping in mind; the treatment^site interaction here-. 

The next model-based comparison involves the two siten which were 
'given the PT treatment. Here^ there is a real impact. The gain among 
the EXP school was 1T.29 poirits, and/that among the CON schools was 7.51 
points. The difference is 3.78 points. The t-value is +8.56; this is 
clearly large enough to be unlikely to be due to fluctuations. 
Furthermore, the net impact is 50% which is large enough to justify further 
exploration. It should be noted that the Total Reading results for the 
PT comparison had a t-value of +4V69 and a net impact of 26%. Thus^ the 
two findings are similar in d together they indicate a possible 

pattern of favorable PT model results. 
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the comparison labeled ''PT-E/TO-E" is one that must be viewed with » 
some caution. It reveals that the difference in gain between the EXP 
students in the PT sites (11.29) and the gain of the EXP students of the 
TO sites (9.38) is 1.89 = 20%. This is large enough to have a t-value of 
+4.7^9, but the net impact is only 20%. This comparison is possibly dis- 
torted by differences between sites in the prevailing growth rates, to 
cope with this, the same comparison that was used with the Reading results 
(see pages III-22-,23) was employed; This comparison yieUs a difference 
of li 16 between the gains of the PX and those of the tO, after adjustment 
for site-torsite differences in growth rate. Thus, the net impact is 
re luced to 13%. This impact is just large enough to be considered non- 
negligible. It clearly is above any necessary cut point in the t-statistic 
having a t-value of 4.32. 

the last cqmparisbn is, as with the total Reading results in Table 
III-3, ah overall comparisoh^pf all four experiiaental schools with all 
four control schools i It indicates that the average gain among the EXP 
schools is 10.10 points and the average gain among the CON schools is 
8.24 poihtis. Thus, the difference in gains is 1.86 points. This again 
has a high t-value, +6*98, but the educational relevance of this impact is 
small, 23% . 

Summary j 

To summarize the results of this analysis of achievement gains, it 
seems reasonable to say that some of the hypothesized differences do 
appear in the data. For achievement in Reading, the incentives ./models 
appear to have had an impact worthy of note only in Oakland. While the 
comparison for the sites receiving the Parent-Teacher incentives model 
indicates an impact on Reading that is large enough to consider further, 
it must be remembered that the effect of the PT model was not similar in 
the two sites where it was employed. The model appears to have worked 
fairly well in Oakland, but not at all in San Antonio. 

For Mathematics, the picture is somewhat clearer. In both Oakland 
and San Antonio, the incentives model (PT) seems to have had a sizable 
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Impact ons achievement gain. However, the TO model sites do not show anj* 
detectable effect of the incentives model they experienced. One possible 
explanation for this difference in the effects of the FT model on Reading 
arid cn Mathematics might be that parents find it easier to understand the 
difficulties a child is, having with mathematics, and to provide him with 
useful help. This is j of course, only a possibility, but perhaps is 
worthy of further investigation. 

Any general conclusions from this project roust be drawn very cau- 
tiously, if at all, since the data indicates considerable site-to-site 
differences in achievement level, gain rates, and responsiveness to the 
models. This perhaps is as important as any other result of the project. 
Furthefmbre, supplementary analyses presented in the next section dis- 
cuss soraie additional sources of "noise" in the data, which tend to reduce 
the definiteness of any conclusions that can be drawn. 



/ 
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Further Analyses of Achievement Test Data 

Introduction 

The results presented in the -^ceding section provide the most direct 
and clear summary possible, of the impact of the incentives models on 
academic achievement. While the mode of analysis Is a. simple one, it is 
as satisfactory as any which might be used, given the characteristics of 
the project design and the data. However, tliere are some additional 
points that should be examined and some methcdoiogical explorations which 
might support the previous results or reveal needed qualifications. Thus, 
some additional analyses were carried out and are described in this 
section. 

The use vl! observed gains in MAT standard score rather than some 
form of adjusted gains such as covariance analysis was adopted after a 
review of the literature and a consideration of the issues involved. Firsts 
the use of observed gains provides no loss in the reliability, as compared 
with any form of adjusted gains, because the reliability of the adjusted 
gains also depends on tiie reliability of the data at each time point. 
More importantly,, the analysis here is of average gains . the mfsasurement 
reliability of which is considerably higher than that of individual gain 
scores. Also, because the interest is on differences in average gain 
only, the whole issue of the dependence of individual gain on the initial 
score of the individual is not the problem. Rather, the question is 
whether the average gain is dependent on the average initial score. If 
that is the case, tiie next question is whether adjustments sh^ be made 
for that dependence. If the answer to that question is in the affirmative, 
then thexfe still remains the problem of calculating an appropriate 
coefficient to carry out the adjustment. 

Ideally, it would be preferable to adjust appropriately for any 
dependence of average gain on average initial score. However, with only 
four comparable units , the data aire hot sufficient to obtain any precise 
estimate of the form of dependence. Hence, a judgmental argument is 
used instead. The conclusion from that argument is that the dependence 
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of average gain on average initial score, between schools within site, will 
be negligible* This conclusion is derived from the following two argu- 
nents* First, the schools being compared begin at nearly the same point 
on the general growth curve of Reading achievement or Math achievement 
and are otherwise quite similar* Consequently, their natural growth rates 
can be expected to be very similar* In other words, there is no conspicu- 
ous, relation between initial score and growth rate arising from this 
source, at the school level of comparison* The "fan spread" will be slight* 
Second, there is no particular reason to expect a sizable regression of 
the two school means within a site to a site mean * That is, the small 
achievement differences between schools within site which do exist are 
probably stable over time* This conclusion is based only on the limited 
evidence cullecl from the districts' past achievement tc: results, but it 
is nonetheless the best interpretation available in the absence of richer 
and more appropriate data* 

So, then, the decision rot to adjust is based on evidence which is 
not conclusive*' However, it is the best evidence available* An additional 
reason for not adjusting is that there is no satisfactory way of deciding 
what value the adjustment coefficient should have* If a coefficient were 
chosen on the data available, and adjustoent were made according to it, 
there would be no way of knowing the direction of the error in the re- 
sulting adjusted score* 

Before going^on to the specific analysis, one further point should 
be made about the whole question of comparing gains in academic ijchieve- 
ment* There are two particular ways in which growth can depend upon 
initial score—one is "fan spread", and the other is regression toward the 
mean* These factors obviously complicate any attempt to control for the 
dependimce of grow di on initial score* There is an additional ccmplicatior « 
however* The two sources of dependence tend to work in opposite direc- 
tions* The Its e of linked interval scores such as the MAT standard scores 
employed in this analysis reduces this problem somewhat for school level 
analysis by counteracting the positive dependence of growth on initial 
score across age levels* In other words, through use ot the standard 
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scores, there can be more confidence that the direction of any small net 
bias in school average gains is to make the initially low scoring group 
show a greater gain. So, vhile there is still some ambiguity present in 
the results, the combination of influences almost certainly will be such 
as to make the final results (if anything) slightly optimistic as far as 
estimates of treatment impact are concerned. 

To pursue this and other matters in more detail, some methodological 
investigations have been carried out. Several topics were considered 
in these further analyses: 

1. The fan-spread phenomenon and ways to deal with it 

2. The regression toward the mean phenomenon and its relevance in 
this analysis 

3. Patterns of achievement gains within each: girade 

4. Specific patterns in the results that can be identified when 
starting and ending scores as well as gains are studied. 

Fan-Spread Analysis , ' 

As discussed prey iot^ly, one problem that has been noted in the use 
of achievement test data is that the amount of gain which occurs in the 
scores of target populations is to some degree proportional to the initial 
score level of those pbpulatioris, even in the aljsence of any treatment at 
ail. Thus, if a study is carried out in which the control group has 
initial scores higher than those of the experimental group, the simple 
comparison of gains made by the experimental grou by 
control group would tend to be biased against finding a positive impact 
from the treatment;. This point was made with slightly different emphasis 
by Campbell Erlebacher (1970) in their review of the Westinghouse-Ohio 
University evaluation of Head Start. It. is also discussed in some detail 
by Campbell in a paper published in the 1970 Proceedings of the Educational 
Testing Service Invitational Conference on Testing Problems. 

The basic approach suggested by Campbell in the Educational Testing 
Service paper is to note the fact that if growth rates are, in the absence ^ 
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of any treatment, proportional to iiiitial score, then the variance of a 
set of scores at Time 2 will be greater than the variance of the same 
scores at Time 1, by the same proportion. In other words, Campbell notes 
that this difference in variances has been observed empirically and he 
makes the connection between it and differential growth rates within the 
groups as well as between the groups. The question then is: How does 
one adjust for these irrelevant differences, so as to be able to obtain 
ah appropriate measure of the treatment's effect? 

The idea suggested by Campbell is to compare not the observed gains, 
which may be affected by differences in the metric (i.e.j different 
standard deviations implying different natural growth rates), but rather 
to compare the difference between the experimental and the control group 
at Time 1 (before the initiation of the treatment) with the difference 
between the experimental and the control group at Time 2 (after the impact 
of die treatment). The important point is that this comparison is in a 
metric which is unaffected by differences in growth rate between the 
^perimental and control groups (assuming that such differences are pro- 
portional to the standard deviations) • 

One metric which accpmplishes this is that of the t- test • That is, 
the differences between the experimental and the control groups at Time 1, 
measured in units of the t-statistic, should be the same as the differaace 
between the experimental and die control groups at Time 2, also measured 
in units of the t-statistic, provided that there is no real trektment 
impact, and regardless of whether the variances at Time 2 are larger than 
those at Time 1. UrifdrtuMtely, Campbell does no provide in his arti- 
cles the exact form of a statistical test whidi might be used to decide 
whether or not the two t-values are equal. However, this is not top 
important, given the data, because the objective is only to get some 
approximate sense of the size of the differences in the t-value. Since 
the values being compared are t-values, it seems reasonable to regard 
differences in t-value of 1.0 to- 2.6 as indicating a small change, those 
smaller than that as negligible, and those larger as being noteworthy. 
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If this comparison is to be accomplished in ah effort to. minimize any 
bias due to fan spread, it is necessary to show the pre and post-test 
school average results for Reading arid Math achievement. Tables III-5 
through lil-8 do this for Reading Pre, Reading Post ^ Math Pre, and Math 
Post, fespectively^-^'^---™^ 

The first point that can be noted upon inspection of these tables, 
which show the standard deviations, case bases, and t-valties as well as 
the school means ^ is that the standard deviations for the post-tests are 
all about the same size as those of: the corresponding pretest, and tlsually 
are slightly smaller. This indicates that the results of the original 
gains comparisons are not likely to be affected by differences in growth 
rates between schools. 

To be thorough, however, the differences in the t-yalues from pre to 
post for both Reading and -Math were calculated and; each difference was 
examined. The results of Uiese separate comparisons, as well as summary 
results from the gains analysis, are presented in Table lil-9. They 
provide no reason to modify any of the conclusions drawn in the main analy- 
sis. 

Regression Effect 

A second ppssible source of distortion in an analysis of this general 
type is the elusive phenomenon knoTO^ as regression toward the mean. 
Basically, this phenomenon can be described by saying that if a set of 
scores is examined, and the units with} low scores are compared to those 
with high-scores, then there will be a tendency for the high scores to 
move downward twafd the mean of the entire set, and for the low scores to 
move upward toward the mean of the entire set; If there is a general 
growth^ process occurring, this regression phenomenon will marii^^ itself, 
as a greater growth among those units i^ich were: initially low in score. 

(Text resmes on page 111^38) 
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TABLE lil-9 
READING 





Original t 
r or vja xns 


Original Net 
iiupac z 


Diff Of 
t s 


CIN 


•^1.69 


-0.11 


-0.49 


TAX 




+0 .0/ 


+u.5y 


OAK 


+5.09 


40.42 


+1.70 


; SAN 


40.76 


40.06 


40. n 


TO 


+0.22 


+0.01 


40.05 


PT 


+4.69 


40.26 


+1.56 


; PT-E/TO-E 


+3.77 


40.17 


+1.57 


E/Ci All 


+3.25 


40.10 


+0.92 




NATHEMATICS 


- - - - - - 






Original t 
for Gains 


Original Net 
Impact 


Diff Of 

u S 


CIN 


-1.92 


-0 ii2 


-0.65 








+0 Q4 


; OAK 


+5.96 


40 .48 


+2.02 


SAN 


+5.30 


+0.44 


+2.29 


TO 


+1.81 


+0.07 


+0V30 


PT 


+8.56 


+0.50 


+3.30 


PT-E/TO-E 


+4.79 


+0.20 


+2.33 


E/C, All 


+6.98 


+0.23 


+2.26 ■ 
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The possible relevance of the phenomenon in this experiment is as 
follows. For most of the sites and most of the grades, the students in. 
the EXP schools had average initial scores somewhat below the average 
scores of the students in the CON schools, therefore, to the extent 
that the regression phenomenon operates between schools, within a site, 
the students in the EXP schools would show greater average gains, regard- 
less of any treatment. In other words, the greater average gains that 
have been regarded as being evidence of the impact of the models may in 
fact be simply the result of a statistical artifact. The problem then 
becomes one of estimating the size of this regression, if possible. 

It is here that the issue becomes complicated. The possible scores 
of the regression effect include all components of the score which are 
stochastically unstable over the time interval under consideration. That 
is, the sources include what is ordinarily classed as measurement error 
and other components as well; In. this instance, the ^portion of the 
regression effect that is due to measurement error will be extremely 
small, because the averages under consideration are based on large num- 
bers; and the measurement error of these averages is trace. However, 
there are. other potential sources of regression effects extant, in the 
foria of temporary influences that operate at the grade or school" level. 
These influences are not well understood ^ as to source or magnitude, 
but appear in general to be top large to neglect entirely. Thus,, a prob- 
lem does exist; Although the regression of individual scores toward their 
school averages will not be involved, the regression of the school aver- 
ages within a site to a common site average Is possible arid will be con- 
founded with the impact of the treatment. 

The reasori that this regression of school averages is confounded with 
the treatment impact is that the schools were not assigned to the experi- 
mental or control condition by a strictly raridom process, but instead by 
an administrative decision -which took into account past achievement 
scores, and poverty levels in the schools and attempted generally to apply 
the treatment in the more deprived school. This decision was iri accord 
with the political realities that attend the allocation of school district 
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resources* This is an unfortunate limitation of most experiments and 
demonstrations of this type, and It restricts seriously the kind of con- 
clusions that can be drawn* If there had been several replications of 
the two treatments wlthlti each site, then some empirical estimate of the 
regression of the school means might have been obtained from the conver- 
gence of die means of the schools within each treatment* However^ such 
data are not available* 

Thus, one is left with no satisfactory way to make a quantitative 
estimate of the size of the regression distortion. In general, it is 
clear that the estimate of the treatment Impacts presented in the previous 
section will need to be lessened, given the possibility of at least a 
small regression of school averages* 

A final consideration in this discussion of regression, while far * 
from rigorous, does seem to have some merit* The Impiemehtation of the 
incentive models was far from optimal. The district grants were delayed 
by three months and^ die agre^wits with the teacher and parents were 
delayed by as much as five months. This is not a criticism, for the logis- 
tical and administrative problem encountered were enormous. The point 
is that the delay in fully implementing the incentives models almost 
certainly attenuated the potential effect of those models* Thus, this 
weakness ih Implementing the models will tend to compensate for the dis- 
tortions introduced by the regression effect. 

The overall conclusion of this discussion of fan spread and regres- 
sion, then, must be that the specific estimat^^ 

reported in the preceding section cannot be regarded as very precise. They 
might well be off the mark and are probably somewhat high. On the other, 
hand, there is no feasible way to make a quantitative judgement as to how 
far they are wrong in any ispecif led case, and so the results presented 
earlier are the most reasonable of any sliig^le estimate. The real point 
to be drawn from this discussion is that firm Inferences about the sizes 
of effects can be made only if appropriate procedures are followed so 
that there is tight execution of a sound research design. 
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Grade Level Comparisons 

The next point to be discussed concerns the results for the achieve- 
ment test within each grade. The incentives models were applied uniformly 
to all grades within the experimental schools. There is no feature of 
the design, nor any clear theoretical justification, for expecting the 
impact of the incentives models to differ according to grade levels. Thus, 
this comparison is not central to the policy issues inherent in the design 
of the project. However, it does have some relevance to the Federal 
interest in early intervention strategies and future field applications 
of incentives. 

For those reasons^ an analysis of the achievement test results was 
carried out within each grade aiid site. These results are summarized in 
table III-IO, for Reading, and Table III-ll, for Mathematics. 

The primary distinction to be made about the grade-by-grade analysis 
is to be clear as to what interpretation is to be given to various possible 
outcomes. First, if there is no grade- treatment interaction in a particular 
site (i.e. J the impact of die treatment is uniform across the grades), then 
that is the end of the matter. Alternatively, it may be that there are 
differences in responsiveness between the grades and that such differences 
have a discernible pattern. For instance, the treatment may have a larger 
impact at Grades 1 and 2 than at Grades 5 and 6. If this were the case, 
there would be clear scientific and policy implications. For policy, the 
implications would be a restricted applicability of the incentives models, 
and for science there would emerge the question of explaining why the 
grades differ in responsiveness. A third possibility is that differences 
will be found between grades in their responsiveness, but that these 
differences will be more or less random. Such a lack of pattern would 
presumably reflect the operation of idiosyncratic factors . 

If the impacts appear to be more or less random, it is reasonable to 
group the sites within the inc«itive models. That is , the set of grades 
being studied (Grades 1- 6) is quite important in its own right—in 
statistical terms it is a "fixed" effect—and there are enough cases in 
each grade so that one can be at least reasonably confident that the 
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TABLE III-IO 
READING 



GRADE 


CIN 


JAX 


OAK 


SAN 


1 


-1.86 


-1.58 


+1.26 


-3.69 


2 


-1.93 


-4.40 


+4.77 


+0.90 


3 


+0.20 


+2.71 


+3.12 


+1.21 


4 


+1.27 


+1.74 


-1.23 


+0.90 


5 


+2.58 


-om 


-0.43 


-0.04 


6 


-1.76 


+1.72 


+3.78 


+2.05 



TABLE III-ll 
MATHEMATICS 



GRADE 


CIN 


JAX 


OAK 


SAN 


i 


+0.45 


-0.54 


+2.74 


+2.68 


2 


+0*53 


-1.60 


+2.19 


-0.24 


3 


-2.46 


+1.86 


+3.00 


+2.17 


4 


-0.77 


+1.43 


+0.66 


+5.87 


5 


+1.96 


rO.42 


+1.37 


+1.24 


6 


-0.37 


+4.79 


+4.56 


+1.35 
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averages over all the grades are stable. In short, the emergence of the 
last pattem--rand6m differences atoong grades in their responsiveness to 
the treatments—would leave some scientific guestions open, but would not 
obstruct the drawing of policy recommendations. 

An inspection of Tables III-IO and III-ll will indicate that indeed 
it is this third pattern of random impacts which emerges in the daU. 
That is, the impact of the treatment is not consistent within the differ- 
ent grades at a given site, but ttiese differences show no particular 
pattern when examined across sites. 

For example, the overall result for Reading in Cincinnati is a negli- 
gible difference in the direction opposite from the hypothesis, table 
III-IO indicates that this arises from a situation in which Grades 1, 2, 
and 6 show negative results, and Grades 3 and 4 show negligible results 
in a positive direction^ and Grade 5 shows definite positive results. 
When the corresponding data for Jacksonville are examined, one again 
finds negative results in Grade 2, and results in a negative direction 
(though niegligible in size) in Grade 1. This slight similarity to 
Cincinnati disappears, however, when the other grades are examined. In 
Jacksonville, Gradf. 6 shows a positive impact, as also does Grades 3 and 
4. Grades (which was a positive impact in Cincinnati) is essentially 
nil in jfacksonville. Further Inspection of the table reyeils no con- 
sistent pattern of iinpacts in any of the sites in either Reading or Math 
achievemer.Ci 

Thus, the grades can legitimately be pooled in order to obtain an 
overall estimate of the impact at the site, and the scientific point of 
differential impact can be noted. This differential impact is reminiscent 
of the site-to-site differences noted in the main analysis, a further 
reminder that there exists a paucity of knowledge about the factors In- 
fluencing educa tio nal prbduc tivi ty . 
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Specific Patterns in the Achievement Results 

The next topic to be discussed in this section i.^ the specific results 
found at each of the sites. As an aid to the comprehension of these results 
(which are presented in Tables III-5 r III-8) » the same results are pre- 
sented graphically in Figures 1 - 16* 

Each figure shows one of the comparisons in Tables III-5 - III-8, for 
the pre and post-tests. 

The specific results vary from site to site and comparison to compari- 
son. For example; consider the fact that one of the FT sites (Oakland) shows 
a sizsi>le impact in Totax Reading, but the other FT site (San Antonio) shows 
no detectable effect. These impact differentials are attributable hot to 
differences in gains made by the experimental schools; rather, they are 
due to the fact that the gain in the Oakland control school was the 
lowest of the four schools under consideration, whereas the gain in the 
San Antonio control school was the highest of the four. This observation 
does not directly explain why this difference occurred; but perhaps it, 
like the other results, can shed some light on the vagaries of field 
research in education. In other words, the significance of the finding 
for achievement gain in Reading should not be exaggerated, because there 
is no way to know what pattern of events led to it. The point here, as 
in the previous analysis, is that these substantial and unaccountable 
variations indicate the need for a much closer study of the process of 
education. 

In Total Math also there are some puzzling patterns. For ex^ple, 
the gain made by the CON school in Cincinnati was: the largest of the 
four CIMJ school gains, but tiie gain made by the EXP school in Cincinnati 
was the smallest of the four EXP school gains. This makes it seem likely 
that the Math treatment might have been positively effective in all 
four sites (though perhaps not very large in effect in some of the sites), 
but for some peculiarity of the situation at Cincinnati^ One possible 
explanation for this anomalous situation in Cincinnati is that the control 
school regarded the project as a direct challenge. That is, having been 
told that a new and better idea was to be introduced, they decided to make 

(Text reames on page IIIt60) 
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FIGURE III-2 
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FIGURE III-3 
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FIGURE III-ll 
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FIGURE I 11-12 
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an all-out effort to show that the old way is in fact better than the new 
one. The results of MAC observations and conversations between TAC staff 
and staff in the Cincinnati control school indicate that this kind of 
motivation was operating to some degree there. Thus, there Is another 
complication that must be considered. 

Sumnary 

In this section, several sources of possible distortion or ambiguity 
in the achievement results have been examined. The general conclusions 
from this examination are that there seem to be no detectable distortions 
of large magnitude. On the other hand, there are several sources of 
anibigulty in the achievement results. 

Because of this ambigui^, it would be unwarranted to make specific 
recomnendations from this study as to what Federal policy should be with 
regard to the use of incentives to teachers and parents as a means of 
i]q>roving educational performance. The conclusions which can be drawn from 
this analysis of the achievonent data are of a different nature. The 
specific conclusions from this further analysis of the achievement data 
is that although the results must be regarded as tentative, there is a 
pattern in the achievement data which suggests that incentives may promote 
increases in learning rate* These increases were most pronounced for the 
Parent*-Teacher model, and for achievement in Math. The pattern does not 
appear to be due to the methodological artifacts of the analysis made. 
However, the caution should be made that these results could be due to 
causes other than the incentives treatment. In fact, the analysis of the 
attitudinal and behavioral data from students, teachers, and parents, 
which is presented in the subsequent sections, will tend to weaken the argu- 
ment that the observed achievement patterns are simply the results of the 
Incentives treatment, because there will be very little consistency between 
the achievement results and the results of these other analyses. 

A second, more general, methodological conclusion emerged from this 
further analysis of the achievement data. There are several analytical 
issues involved in the use of achievement test data for this kind of 
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purpose. The possibility of floor and ceiling effects on test scores was 
recognized by the project designers^ and the use of the multilevel test 
administration system was an attempt to deal with it. Similarly^ the 
use of carefully trained and supervised test administrators and explicit 
procedural instructions was an attempt to minimize the potential lack of 
reliability and differential reliabilities. However, these testing pro- 
cedures have not been empirically validated with these data; therefore, 
at this point these are no more than assumptions about how tests ought 
to be assigned and administered. Furthermore, additional technical ques- 
tions remain: the choice of the most accurate score format in multilevel 
testing; the more general problem of regression effects o£ school means; 
and the fan-spread phenomenon. The latter two issues have only recently 
been recognized, and far more research into them must be carried put if 
educational research is to inform policy makers wisely. 
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Analysis of Student Attitudes and Behavior 

Introduct!<nn 

The analysis of the questionnaire, interview, and observation data 
presents somewhat different challenges than those posed by the achievement 
data. For example, with the achievement data, it was necessary to make 
some decision as to score format, but the content of the scale was esta- 
blished by the publisher. For the questionnaire, interview, attendance, 
and observation data, however, the decisions as to index construction had 
to be made during the analysis. Circumstances of the project, particularly 
the very tight time schedule and the problems of obtaining clearances 
for the instruments, coupled with the fact that the unusual nature of the 
project made it possible to adopt existing instruments, precluded any 
full scale program for the validation of the survey instruments. Instead, 
special ad hoc instruments were developed which asked the relevant ques- ' 
tions in a direct and simple way. 

The content of these instruments was determined by the hypotheses 
that had been formulated concerning the potential impacts of the incentives 
models. For example, one group of hypotheses asserted that students in 
the EXP schools would become more favorable in attitude toward several 
aspects of the school experience, m particular, it was hypothesized 
that students in the EXP schools would become more favorable toward 
(1) reading school work. (2) arithmetic school work. (3) teachers, and 
(4) school in general. An additional hypothesis was that students in 
the EXP schools would become more optimistic and confident about further 
school experiences, and would develop stronger aspirations for high 
educational attainment. Also, it was hypothesized that students in the 
EXP schools would become more concerned about the performance of their 
fellow students, and more favorably disposed to assist them. Finally, 
it was hypothesized that students in the EXP schools would develop more 
favorable self-images and perceptions of their achievements and oppor- 
tunities. This last is a deliberately broad concept, embracing some 
elements in the concept of orientation toward future schooling as well as 
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some direct self-image elements. These hypotheses led to the development 
of items for the questionnaires and interviews which would tap each 
variable described above. Such items were included both in the Student 
Questionnaire and Student Interview Schedule. A similar procedure was 
followed in developing the data collection instruments for use with 
teachers and parents. 

The instruments were designed to be easy to administer. As a result 
of these efforts, and the careful planning for the administration, the 
completion rate for items within questionnaires and interviews for this 
project is much higher than the usual level encountered in studies dealing 
with disadvantaged elementary students. However, another consequence 
of the necessary compromises in the development procedure is that the 
instrtunents have only face validity (which is quite high, given the 
decision to make the questions simple and direct) , and have unknown 
reliability, 

A second issue with respect to the attitude and behavior data, and 
one which applied to parents and teachers as well as to students, concerns 
the timing of the first wave of data collections. For the student inter- 
views and the classroom observations, the first wave of data was collected 
from the middle of November to the end of December. Although this was* 
substantially after the beginning of the school year, it can still be 
regarded with some assurance as a "pre" measure, since the data were 
collected before the incentives models had exerted any major impacts on 
the schools. In fact, the local districts did not actually sign grants 
to participate in the study until January, and the parents and teachers 
did not have contracts to sign until February and March. All of the data 
from the interviews (student, teacher, and parent) had been collected 
prior to that time. 

However, for the student (and also the parent and teacher) question- 
naires^ the first wave of data was not collected until late March, which 
is quite late in the year. The problem that this creates for analysis 
is that it thus becomes quite problematic whether this administration should 
be regarded as being before the treatment, or during it. In other words, 
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although there are two waves of questionnaire data, it is not at all clear 
that they can be analyzed as. a before-after comparison. For example, if 
the data were to be considered as reflecting the before and after condi- 
tions, and analyzed that way, and if no differences (impacts of the treat- 
ment) were found, what conclusion could be drawn? One analyst might con- 
clude that the treatment did not have any impact, and another might decide 
that perhaps after all the impact had occurred before the first (March) 
measurement. 

Obviously, in this situation, the interpretation of each result must 
be made somewhat subjectively, and on the basis of the relevant data for 
each scale and each comparison. Consequently, .the results are presented 
in tables that show, for every index, the average of the EXP and CON at 
the first administration, and the average of the EXP and CON at the second 
administration. 

Even under these difficult circumstances, however, there are some 
possible patterns of outcome which seem unambiguous. For example, if 
there is a difference favoring the experimental group at the second 
administration, but a considerably smaller difference at the first adminis- 
tration, then (except for possible regression effects) one can rather 
safely conclude the incentives treatment had an impact which occurred 
during the interval between the first and second administration of the 
instrument. If, on the other hand, there is a difference on the second 
administration favoring the EXP group, but that same difference exists 
at the first administration, the conclusion cannot be definite. The 
analyst (and the reader) must then decide how likely it is that the dif- 
ference found at the first administration represents a pre-existing dif- 
ference, and how likely it is that it represents instead a quick impact 
of the incentives treatment. Since the schools within a site were not 
assigned at random to an EXP or CON treatment, there is no way of arguing 
that any pre-existing differences will be dampened out when sites are 
combined. The only guideline available in this situation is that one must 
be consistent in going from item to item or index to index. If the con- 
clusions drawn from the results of one index are inconsistent with the 
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conclusions drawn from the results of another index, then the only prudent 
reaction is to suspend judgment* 

If there are no differences at the second administration favoring 
the EXP group, or perhaps even a difference favoring the CON group, then 
again the situation may be indeterminate. The xesults of all this analy- 
sis, in short, are dependent upon the judgment of the reader • This is an 
unavoidable consequence of the design and execution of the study* These 
results should thus be regarded as providing not firm knowledge about the 
impacts of the incentives models, but rather as indicating promising 
areas for future investigation* 

Measuring the practical magnitude of an impact under these conditions 
is of course doubly difficult, since even the numerical value of the impact 
is not certain. However, in instances where there is some reason for 
confidence in the real existence of an Impact, the size of that impact 
in substantive (educational) terms will be estimated as closely as possible* 

Regression of scores toward a common mean is more likely with these 
survey items than it was with the achievement test results, since these 
items are used individually or in indexes based on only a few items, rather 
than being scores based on the sum of a large number of items* In other 
words, unreliability is more of a problem with these survey items than it 
was with the achievement test scores* 

As with the achievement results, there are always Hawthorne effects, 
John Henry effects (Saretsky, 1972) and possibly the opposite of a John 
Henry effect, what might be called the Rosenthal effect (Rosenthal,1966) * 
There is, however, no particular reason to expect a fan*spread phenomenon 
to be occurring on these survey items. 

Index Construction 

Data on student attitudes and behavior were collected from four 
sources, at several points in time* The sources included: 
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1. A student questionwire which all 4600 students were asked to 
complete 

2. A student interview which was completed by a random sajnple of 
approximately 200 students 

3. Attendance records for each classroom each month 

4. Classroom observations on student behavior, collected within 
each of the approximately 160 classrooms of the project. 

Because all of this data provides information about the impacts on students 
as a result of the incentives project, all of it will be discussed in 
this section. Subsequent sections discuss the parent and teacher results. 

For the Student Questionnaire, there were 43 items. These were pre- 
sented in such a way that reading skill was not essential to completing 
the task. The first 13 items were coded as 1 or 2 or 3, with the higher 
number always associated with the response that appeared most desirable. 
The remaining items were coded 1 or 2, with the higher number again 
always associated with the response that appeared to be most desirable. 
Thus, the indexes created by addition of responses always were oriented 
so that a higher score meant a more favorable outcome. From these 43 
items, seven indexes were constructed, based on the research hypotheses 
and the item content. These indexes were created by adding together the 
responses to each of the component items. If a response to any item was 
missing for a particular student, then the value of the index also was 
considered to be missing for that student. Since the completion rate 
for items on the Student Questionnaire and interview was very high 
(averaging over 95%), it was possible to follow this simple and desirable 
strategy without losing a substantial amount of data. The names of the 
indexes constructed from the Student Questionnaire items, and the items 
which constitute each, are shown in Table III-12. 

For the Student Interview, there were 15 items. From these 15 items 
six indexes were constructed. The indexes parallel those developed from 
the Student Questionnaire; however, there was no index of ' "Orientation 
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TABLE III-12 

INDEXES CONSTRUCTED FROM THE STUDENT QUESTIONNAIRE 



!• Student Attitude Toward Reading 
Question No, 



1 When I think about learning to read 

3 When I think about reading 

5 When I think of my reading teacher 

15 Reads well vs does not read well 

18 Likes to read at school vs^ does not like to read at school 

20 Reads alot at home for fun vs_ watches TV or plaj's 

21 Goes to a library to re;id for fun vs does not go to a library 

2. Student Attitude Toward Arithmetic 
Question No> 

2 When I think about learning to do arithmetic problems 

4 When I tiiink about doing arithmetic problems 

6 When I think of my arithmetic teacher 

14 Good at doing arithmetic problems vs not good at 

17 ■ Likes to work at school witli arithmetic problenis vs does not 

22 Does arithmetic problems at home for fun vs watches TV cr plays 
28 Wants to be good at doing arithmetic problems vs does not care 

3* Student Attitude Toward Teachers 

Question No, 

5 When I think of my reading teacher 

6 When I think of my arithmetic teacher 

8 When I talk to my teacher in school, the teacher looks 

10 When one of the kids asks a question, the teacher looks 

12 When we try to liave fun in class; our teacher looks 

4. Student Attitude Toward School 
Question No > 

13 Goes to a good school vs goes to a bad school 

16 Likes to go to school vs does not like 



5. Student Orientation Toward Future Schooling 
Question No* 

27 Wants to do well in school vs does not care how well he does 

33 Expects to go to high schooJ vs does not expect 

38 Wants to go to high school vs_ does not v/ant to go 

39 Wants to get training for a good job V£ does not care 
Expects to be trained for a good job vs^ does not expect to be 



41 



- Continued 
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TABLE III-12 (Cont'd.) 



24 

25 
26 



6. Student Relations with Other Students 
Question No. 

9 When other kids ask me for help in their school work, J. feel 

11 When another kid in my class makes a mistake, I feel 

Help otlier kids outside of school with their schoolwork vs does 
not 

Helps other kids at school with their school work vs does not 
Wants all the kids to do well in school vs does not care 

7. Student Orientation Toward Self 
Question No. 

14 Good at doing arithmetic problems not good at 

15 Reads well vs^ does not read well 

27 Wants to do well in school vs does not care how well he does 

28 Wants to be good at doing arithmetic problems vs does not care 

29 Wants to be a good reader vs does not care 

30 Most people think child is good vs most people think child is 
bad 

31 Expects to be a good reader vs does not expect to be 

32 Easy to learn things vs hard to learn things 

33 Expects to go to high school vs does not expect to go to high 
school 

34 Can do the things he wants vs cannot do the things he wanes 

35 Expects to be good in arithmetic vs does not expect to be good 

36 Finishes his work vs gives up easily on his work 

37 Likes himself most of the time vs does not like himself 

40 Finishes his work vs will get what he wants because of luck 

42 Will get the job he wants vs will not get the job he wants 
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toward Self 'developed from the Student Interview data. The same procedures 
were followed in constructing the Student Interview indexes as had been 
used in constructing the indexes for the Student Questionnaire data. It 
should be noted, however, that the number of cases for the questionnaire 
responses is approximately 3800, and the number of cases for the interview 
respor*ses is approximately 200. The names of the indexes for the Student: 
Interview items, and the items from which each was constructed are 
shown in Table III-13. 

Because the content, of these two instruments is similar, and because 
the indexes refer to attitudinal rather than behavioral dimensions, it 
seems best to discuss together the results from the Student Questionnaire 
and Student Interview data. After that discussion is completed, a 
discussion of the attendance data and the classroom observation data on 
student behavior will follow this presentation. 

Single-Site Comparisons — Student Questionnaire 
and Interview Results 

The first comparison examines the results of the Student Questionnaire 
and Student Interview data at Cincinnati, and compares the EXP and CON 
schools there. Cincinnati is one of the sites at which the Teacher-Only 
(TO) model was used. Tlie results for the Time 1 and Time 2 administrations 
are presented in Tables III-14 and HI-'lS, respectively. The general 
picture in the Cincinnati data on student attitudes is fairly clear. For 
Cincinnati, there are few changes in t-value between the first and second 
administration of the instrument that are worthy of note. Interestingly, 
the few change^ ie.g.^ in "Attitude toward Arithmetic" and "Relations 
with Other Students" from the Student Questionnaire, and in "Orientation 
toward Future Schooling" from the Student Interview) are in the directioiT 
that indicates change toward less favorable attitudes among students in 
the EXP schools. However, two of these changes (those from the Student 
Questionnaire) might be due to regression effects, since students in the 
EXP school were initially very high on these indexes . The t-value for 
the Student Interview index of "Orientation toward Future Schooling" at 
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TABLE 111-13 
INDEXES CONSTRUCTED FROM THE STUDENT INTERVIEW 



1. Student Attitude Toward Reading 
Question No. 



4 
5 
9 

13 



Do you enjoy work in reading, or would you rather not do it 
How many times do you visit the library on your own.out of class 
Would you like to be a good reader 

Do you expect to be a good reader when you leave school 



Student Attitude Toward Arithmetic 
Question Ko. 



8 

14 



How do you feel about doing work in arithmetic, do you enjoy it 
or woul-5 you rather not do it 

Would you like to be good at doing arithmetic or <?on't you care 
Do you expect to be good at arithmetic when you leave school 



Student Attitude Toward Teachers 
Question No. 



11 
15 



Does your teacher give you a hard time if your work is not good 
How do you get along with your teacher, well, or not so well 



Student Attitude Towara School 
Question No. 



1 
2 



Do you enjoy school. Tell me how you feel in the morning be- 
fore coming to school — glad or not so glad 
When you get to school, do you like the work you do in class 



Student Orientation Toward Future Schooling 
Question No. 

10 When you finish school, do you expect to get a good job 

Student Relatio;.8 with Other Students 
Question No. 



6 
7 



When you do your school work, do you get any help from your 
friends 

Do you help your friends with their school work 
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Time 2 is not calculable, since there was no wi thin-group variance on 
this index for the CON school* However, the difference between the EXP 
mean and the CON mean at Time 1 is +.0O59. The difference between these 
two means at Time 2 is -.2778. Thus, the Time 2 difference is in the 
direction favoring the CON group, and is not negligible in size. For 
Cincinnati generally, as far as students self-expressed attitudes and 
orientations are concerned, there is only one t-value above 2. DC in magni- 
tude. This final t-value greater than 2.00 occurs on the Student Question- 
nair index "Attitude toward Teachers" presented in Table III-15, Time 2. 
It indicates a less favorable attitude toward teachers exists in the EXP 
school than in the CON school at the second time. The same direction was 
also observed when the first wave questionnaire was administered. This 
pattern is opposite in direction to any possible regression effect. 

Turatng next to Jacksonville, one finds a different and somewhat 
puzzling pattern. Here, four of the Questionnaire-based indexes show 
changes that seem to indicate favorable impact of the treatment. The treat- 
ment used at Jacksonville was the Teacher-Only model. The results for 
Jacksonville are presented in Tables III-16 and III-17. The four indexes 
from the Student Questionnaire which show changes in the predicted direc- 
tion are (1) "Attitude toward Arithmetic", (2) "Attitude toward Teacher", 
(3) "Attitude toward School", and (4) "Relations with Other Students'*. 
Another index, "Attitude toward Reading", tends in the same direction, 
although less strongly than the first four mentioned. However, on two 
other Questionnaire-based indexes, namely "Orientation toward Future 
Schooling" and "Orientation toward Self", there is a clear change favoring 
the students in the CON schools. In the cases of "Orientation toward 
Future Schooling", there is no difference at the first administration, but 
a difference favoring the CON schools appears at Time 2. In the case of 
the "Orientation toward Self", there is a difference favoring the 2XP 
schools at Time 1, but that difference has disappeared at Tine 2. This 
latter pattern could be due to regression toward the mean. It should be 
noted also that the EXP school in Jacksonville has, judging from the 
Student Questionnaire data, a more favorable general climate among its 
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students, at both times, than the CON school. The only exception to this 
is the "Orientation toward Future Schooling" index. 

The puzzles for Jacksonville appear when the Student Interview results 
for the corresponding indexes are considered. For four of the six Inter- 
view-based indexes, there are no changes at all. For the remaining two, 
on the other hand, there is some evidence of change. The paradoxical aspect 
of this change is that it moves in the opposite direction from the change 
observed for the corresponding index based on the Student Questionnaire. 
For instance, the Questionnaire-based index for "Attitude toward Teacher" 
is higher for students in the EXP school at Time 1 (t = 7.21) and the gap 
becomes larger in favor of the EXP school at Time 2 (t = 8.45). However, 
when the- corresponding index based on the Student Interview data is 
examined, it is found that the Tioie 1 results show a difference slightly 
in favor of the EXP school (t = 1.15), and that the Time 2 results show 
no difference at all (t = O-.Ol). A similar paradox occurs with regard 
to the "Orientation toward Future Schooling" index. The index based on 
the questionnaire data changes from a negligible positive t-value (+0,47) 
::o a non-negligible negative t-value (-1.71); the corresponding index 
from the Interview results changes from marginal negative (-0.99) to 
marginal positive (+0.81). These results remain puzzling, and prevent 
the drawing of any clear conclusions as to the impacts of the incentives 
treatment upon student attitudes in Jacksonville. 

Oakland is one of the sites at which the Parent-Teacher model was 
used. The results in Oakland exhibit an interesting pattern. These 
results are presented in Tables III-18 and III-19. The data from the 
Student Questionnaire indexes show several indications that the PT treat- 
ment is having a beneficial effect. The difference between EXP and CON 
on the Questionnaire-based indexes of (1) "Attitude toward Reading", 
(2) "Attitude toward Arithmetic", (3) "Attitude toward School", and (4) 
"Relations with Other Students" is in favor of the EXP school, and that 
the t-value for each of these indexes is larger at the second administra- 
tion than at the first. This would suggest a positive impact of the PT 



III-76 



CO 



CO 



H 



00 M 



H O 

H 

CO 



CO 











\o 


in 




o 


CO 


o 


j 

1 a\ 


00 


O 


CO 


eg 


00 










CO 




rH 






CM 




XV 




CO 


CM 


in 






U 


rH 


o 


o 


o 


o 


H 


o 


1 ^ 


eg 


o 


CO 


CO 


o 


CO 






• 


• 


• 


• 


• 


• 


• 




















t 

T 


t 

T 




i 


t 

T 




i 


1 + 


+ 




+ 


+ 


+ 


H 
































H 




















1 












CO 
































H 




















1 
















• 




























g 






in 


o 


rH 






CM 




1 ^ 


eg 


eg 


eg 




in 


H 






CO 


rH 






o 


CM 


in 




rH 


o 




in 




CO 




> 


cn 


rH 


rH 


o 


9 




9 


CO 


e4 


o 


eg 


eg 


9 








+ 


+ 


+ 


+ 


+ 


+ 


+ 


1 + 


+ 


1 


+ 


+ 


+ 
















CO 


ON 


o 


1 ^ 




vO 




in 












CO 










H 


CM 


CM 


eg 


eg 


eg 


eg 








tn 


UO 


m 


in 


in 


in 


in 






















































00 


a\ 


o 


o 


o\ 


in 




00 




in 


o 


in 




H 


• 


00 


o 




vo 




in 


CM 


rH 


CO 






o\ 


eg 




s 




rH 






o 


o 


H 


CM 


CM 


eg 


rH 


rH 


o 


eg 








































so 


lO 






- o 


H 




00 


o 




o\ 


eg 


o 








^v 












CO 


1 


o 






CO 


in 


CO 




•d- 




rH 


CO 


o\ 




as 


<f 


eg 


00 




^ 


<t 


H 






rH 


rH 


rH 








eg 


rH 


H 










RESU 




































o\ 


>^ 




o\ 


00 


CM 








in 


in 


m 


m 








CO 




o\ 


o\ 


o\ 


o\ 


vO 


1 CO 


CO 


CO 


fO 


CO 


CO 








cn 


CO 


CO 


CO 


CO 


CO 


CO 


















































• 


o 




o 


00 




O 






CO 


00 


o 


CO 


00 




a 






o 


eg 


m 


vO 


in 


rH 


CM 


o 


00 










CO 


rH 


eg 


eg 


o 


O 


H 


eg 


H 


CM 


rH 


rH 


o 


eg 










































































o 


00 


00 


O 




vo 


CO 


rH 




VO 




o 






IX 


rH 


00 


rH 






o 


sr 


O 


CM 


fH 


00 




00 








m 




eg 


CO 


o\ 


o 


o\ 


vO 


CO 


00 


00 






. 1 






fH 


rH 


rH 






rH 


eg 


rH 


H 











M CO 
O C4 

o w 



CO 



< PS H I 



CO 



w 

CO i 

cy o* o* cy o* cy cy I 

CO CO CO CO CO CO CO I 



M CO 

o cm 

O M 

g g 

CO O 



M M :3C 

o 5 

3: H ^3 
H CO 



j3 



CO CO M CO CO 



ERIC 



111-77 



CO 



CO 

w 



U4 



Pl4 





CM 




M 






iH 
1 




M 




M 




H4 


w - 












<: < 
















l-H O 



CO 



CO 



CO 



CO 



CO 



CO 



IX 



M • 

ptJ CO 

w 
w 



X 



00 CM CN 

m o> CM 



00 o% <r 

CM 



O O O O rH O 
+ + + + + + + 



cn 



o> 00 



o> 00 vo CO 00 vo cn 



CM 



CM 



+ + + + 



+ + 



CM 



m 00 



00 



m m m m m m m 



00 



m CM cn cn 



CM 



cn 
+ + 



o 


m 


iH 


o 


iH 




vO 


00 


-;r 


o> 




o> 


CM 


o 


r>i 


o 


o 




+ 


+ 


1 


+ 


1 


? 


m 


VO 


m 


m 


VO 


m 


CM 


CM 


CM 


CM 


CM 


CM 



CM 



VO 0> O O 00 

cn vo 00 vo 



CM CM 



CM 



VO 



CO o^ 







VO 


m 


VO 


m 


CM 


j 00 


VO 


o 


VO 


00 


-0- 


<r 


<r 


iH 


CO 


a\ 


a\ 




1 


CO 


o> 


00 






rH 


iH 


iH 








CM 


1 


rH 










O 


O 


o 


CO 


rH 


a\ 


! CO 


CO 


CO 




<r 


m 


o 


o 


iH 


iH 


o 


rH 




1 ^ 


CM 


CM 


CM 


CM 


CM 












<r 


CO 














a\ 


CO 


o 


CO 


CO 




! 


rH 




CM 


CM 


o 




OS 




VO 




m 


o 


1 ^ 


<r 


o 


rH 


0^ 


VO 


rH 


rH 


CM 


o 


o 


rH 


CM 


! 


rH 


CM 


rH 


O 


CM 



VO VO 



rH 00 
VO VO 



CO o\ 



as 

CM 



CO 

w 



w h4 - w :s 



M < S 
OtJ W 



H h4 h4 

^ 

CO CO 

o* o* o* o* o* o* o* 

CO CO CO CO CO CO CO 



ON 



VO 0> 
O 00 



CM 



VO 



VO 



w 



CO 



8 g 



CO 



00 



1-4 CO 



M M M M 



CO CO CO CO CO CO 



III-78 



treatment on several student attitudes in Oakland, although some 9£ the 
changes are not large • All of the changes, moreover, are in a direction 
opposite to any regression effect that might be operating* 

When the Student Interview indexes are examined, however, the inference 
that there is a positive effect of the treatment is hardly supported. In 
the Student Interview results, there was only one of the six indexes for 
which even the direction of change was positive, and for that index the 
change is very small. Moreover, for four of the Interview-based indexes, 
the size of the negative change is substantial. This pattern, much like 
that for Jacksonville, precludes any clear inferences about the Impact 
of the incentives model in Oakland, as far as student attitudes are con- 
cerned* Perhaps it should be noted that the shifts in direction on the 
Student Interview indexes are in the direction that would be predicted 
from a regression effect explanation, but this cannot really solve the 
puzzle* 

One consistency between the two student attitude instruments in Oak- 
land occurs on the "Attitude toward Teacher" index* The index based on 
the Questionnaire initially favors the EXP school (t-value at Time 1 is 
+l»4l) and changes to being nearly equal for the two schools (t-value 
at Time 2 is +0.67)* The corresponding index based on the Student Inter- 
view shows a difference of essentially zero at Time 1 (t-value is -0.02). 
At Time 2, however, there is a definite difference favoring^ the CON 
school (t-value is -2.41). This cannot be entirely explained by regression 
effect, since in the Interview data, the change is opposite to what a 
regression effect would predict. 

The fourth of the individual site comparisons based on the Student 
Questionnaire and Student Interview data is that for San Antonio. These 
results are presented in Tables III-20 and III-21. This was, like Oakland, 
a site at wKich the PT model was used* Here, as with Oakland, the results 
seem erratic* They indicate, if anything, a generally negative impact of 
the treatment* In the Questionnaire-based index for "Attitude toward 
School", there is a change from a negligible difference at the first 
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administration between EXP and CON (t-value is +0.71), to a definite 
difference favoring the CON school at the second administration (t-value 
is -3«11). Another index from the Questionnaire which displays this same 
general pattern (a change from a slight difference at Time 1 to a differ- 
ence favoring the CON school at Time 2) is the "Relations with Other 
Students" index. However, for this index, the change is small. The t- 
value at Time 1 is --1.43, and the t-value at Time 2 is -2.90. 

The Student Interview data for San Antonio show a difference on the 
"Attitude toward School" index that favors the CON students, and the 
difference is stable over time. The t-value at Time 1 is -2.11, and the 
t-value at Time 2 is -2.28. The index for "Relations with Other Students" 
from the Student Interview shows no differences between the EXP and the 
CON schools at either time (.the Time 1 t-value is -0.24, and the Time 2 
t-value is -0.61). Of the remaining indexes, those for "Attitude toward 
Reading", "Attitude toward Arithmetic", and "Attitude toward Teachers", 
all show differences in favor of the CON group at Time 1 (or no differences) 
and those differences are increasingly in favor of the CON schools at 
Time 2 although in varying amounts. These changes are opposite in direc- 
tion to a regression effect. Thus, the general picture for San Antonio, 
though fuzzy in detail, seems to indicate an impact on student attitudes 
opposite to that predicted. 

Model-Based Comparisons — Student Questionnaire 
and Interview Results 

In view of the generally inconsistent patterns obtained from' the 
comparisons made at each of the individual sites, the interpretability of 
the model-based comparisons is lessened. However, the results of those 
comparisons are presented, with minimal discussion, since they are the 
best available sximmary information as to the overall impact of the dif- 
ferent incentives models on the various student attitudes. 

The results of the comparison for the pooled Teacher-Only sites 
(Cincinnati and Jacksonville) are presented in Tables III-22 and III-23. 
The Questionnaire-based index "Attitude toward School" changes in the 
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predicted direction, and in a pattern not attributable to regression effect. 
The t-value at Time .1 is +2*57, and the t-value at Time 2 is The 
few other changes that were observed are negative in direction, and po** 
tentially explainable as due to regression effect. 

The model-based comparison for the pooled Parent-Teacher sites 
(Oakland and San Antonio) is presented in Tables III-2A and III-25. These 
results resemble the preceding ones in their lack of consistency. There 
are two positive changes observed in the Questionnaire-based indexes. These 
occur in "Attitude toward Reading" and in "Attitude toward Arithmetic". 
However, for these same two indexes on the interview, the direction of 
change is negative. This negative change might be due to a regression 
toward the mean, since in each case the shift observed is a movement toward 
decreasing differences. That is, for the Interview-based index of 
"Attitude toward Reading", the Time 1 t-value is +1.26 and the Time 2 
t-value is -0.27. Similarly, for the Interview-based index of "Attitude 
toward Arithmetic", the Time 1 t-value is +1*17 and the Time 2 t-value 
is -0.17. The Interview-based index of "Attitude toward Teacher" shows 
a shift toward more favorable attitudes in the CON group. Here the Time 
1 t-value is -0.05, and the Time 2 t-value is -2*30. Thus, this shift 
cannot be attributed to regression effect. However, the corresponding 
index from the Questionnaire shows a difference in favor of the Ul? 
schools which is nearly identical at both times* The Time 1 t-value for 
the Questionnaire-based index of "Attitude toward Teacher" is +3.48, 
and the corresponding Time 2 value is +3.49* Thus, another inconsistency 
between the two instruments emerges. 

The results from the comparison of the EXP schools in the PT sites 
with the EXP schools in the TO sites are presented in Tables III-26 and 
III"27. These tables reveal that there are a number of positive changes 
from the Questionnaire-based indexes, and two negative changes from the 
Interview-based indexes. The two negative changes on the Interview results 
(for "Attitude toward Reading" and for "Relations with Other Students") 
run directly counter to the changes on the corresponding Questionnaire- 
based indexes. However, for this comparison, any of the differences may 

(Test resumes on page III-90) 
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vjell be due to differences in historical events at the different sites, 
and so should net be definitely associated with the differential impact 
of the Parent-Teacher model. 

ITie final set of comparisons based on the Student Questionnaire and 
Student Interview data is the comparison in which all the EXP schools 
are pooled and compared with the pooled COK schools* This cor*parison 
suppresses any possible differences, be t\7een sites or differences between 
the two incentives models. The results from this overall comparison are 
presented in Tables III-28 and III-29 . They indicate some positive change 
in the Questionnaire-based index of "Attitude toward Reading", and some 
negative change in the Questionnaire-based index of "Orientation toward 
Self". In addition, this overall comparison indicates that there are 
several negative changes in the Interview-based indexes. The indexes for 
"Attitude toward School", I'Attitude toward Reading", "Attitude toward 
Aritluiietic"i and "Attitude tox^ard Teachers" all show changes in the direc- 
tion opposite to prediction. Some, but net all, of these changes can be 
explained by possible regression effects. 

Discussion — Student Questionnaire and 
Interview Results 

Perhaps the most interesting outcome of this analysis is not the 
information which emerges as to the particular pattern of student attitudes 
and attitude change at the different sites and under the different treat- 
ments, but instead the high degree of inconsistency between the data col- 
lected from the Student Questionnaire instrument and the Student Interview 
instrument. This lack of consistency makes it impossible to draw sub- 
stantive conclusions from this data with any confidence, but does 
indicate two obvious points of caution that must be noted in any effort 
to carry cut further studies of this general type. The first point is 
technical, and simply stated, is the following: The results of attitude 
surveys are dependent, to a fairly high degree, upon the instrument and 
the circumstances of its administration. Thus, if clear and precise 
comparisons are desired, it will be necessary to invest considerable energy 
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in developing the survey instruments, ascertaining their sensitivity to 
random influences, and then minimizing these influer4ces in the actual data 
collection. The second point is procedural: Field projects of this type 
require large amounts of lead-time for design and the exercise of authority 
to ensure that the original design is followed rigorously. 

To make this point somewhat more specific, we can simply list some 
of the sources of possible inconsistency between the results from the 
Student Questionnaire and the Student Interview* These sources include: 

1. Differences in the content of individual questions 

2. Differences in the format by which corresponding questions are 
presented 

3* Differences in the administrative circumstances in which the 
data is collected (The most obvious such difference in this 
data is that of the time difference between the two administra- 
tions* Other differences include differences in the research 
worker's personality, the physical surroundings, etc.) 

4* An error in the data processing, at any of the several stages 
involved. 

In addition to these technical possibilities for introducing distor-- 
tion into the real pattern, there are two important and fairly plausible 
substantive possibilities* These are: 

1. The students* opinion and attitude on these topics may in fact 
change rapidly, depending mainly upon recent events. 

2* The respondent (student) really does not have a response that 
truly describes him on these topics. They simply are rot 
matters on which he has developed an orientation, yet he feels 
constrained to provide an answer* 

Since the respondents here are fairly young students, these two possibili- 
ties cannot be dismissed without investigation. However, such investiga- 
tion has not been carried out by previous workers, and was not feasible 
with the data and time constraints of the present project* Thus, these 
issues remain unresolved* 
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Student Behavioral Results—Introduction 

The student behavioral data consists of material obtained from school 
documents, namely the average classroom attendance rates, and material 
collected by research workers during structured classroom observations. 
Tlie attendance data will be discussed first. 

The classroom attendance data collected was coded for monthly 
summaries for each full month from October, 1971, through April, 1972, 
the full months during which the incentive project was in operation. 
For each month, in each classroom, three variables were created: 

1. Total number of student days of absence per month 

2. Average number of students enrolled for the month " 

3. Number of instructional days in the month. 

These data were put into the following formula to calculate an index for 
each month: 

/a>senteeism index = total number of absences observed 

maximum possible number of absences 

(variable 1) 



(variable 2) (variable 3) 

These indexes then were used in the analysis. It should be empha- 
sized that the unit here is the classroom, not the student. Thus, a 
figure of 7.36 in the tables means that the absence rate was 7.36% for 
that set of classrooms in that month. It should also be noted that the 
direction of differences in the tables is numerically consistent with 
tables in the other sections. However, since low absence rates are 
more favorable than high ones, it is a negative t-value that indicates 
a difference in favor of the EXP school. Finally, summary indices of 
absence during the fall term were constructed simply by adding together 
the absence rate for November and December. Similar summary absence 
rates for the spring term were constructed by averaging together the 
absence rates for March and April. The results of the analyses- of attend- 
ence are first presented separately for each of the four individual sites. 
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Student Attendance Results — Single --Site Coroparison 

The results of the attendance analysis for Cincinnati are presented 
in Table III-30. They indicate that in the fall term, the absenteeism 
at the EXP school was lower than at the CON school, but that in the spring, 
this difference had disappeared* In fact, in the spring, the absenteeism 
in the EXP school was higher than that of the CON school. This, then, 
seems to be an effect, though small in magnitude, in the direction oppo- 
site to that predicted by the hypothesis* However, this trend is in 
accord with the trend of the achievement data in Cincinnati* Inspection 
of the month-by -month results provides no grounds to qualify the con- 
clusion further* 



TABLE III-30 
ABSENTEEISM DATA—CINCINNATI 



MONTH (S) 


ABSENTEEISM (%) 


DIFF (%) 
(EXP-CON) 


t-val. 


DIFF Ii-I 
FAVOR 


EXP 


CON 


November 


6.86 


7.21 


-0.35 


-0.34 


EXP 


December 


6.94 


9.99 


-3.05 


-2.67 


EXP 


January 


10.69 


9.48 


+1.21, 


+0.84 


CON 


February 


9.17 


9.52 


-0.35 


-0.28 


EXP 


March 


9.04 


8.79 


+0.25 


+0.18 


COK 


April 


9.81 


7.48 


+2.33 


+1.67 


CON 


Fall Term 


6.90 


8.60 


-1.70 


-1.81 


EXP 

1 


Spring Term 


9.43 


8.14 


+1.29' 


+0.99 


CON ; 
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Next, the results of the analysis of attendance patterns in Jackson- 
ville are examined. These results are given in Table III-31. They indi- 
cate no consistent impact of the treatment on attendance. That is, in 
the fall term, the rate of absenteeism at the EXP school is slightly 
lower than the rate at the CON school. In the spring, this same direction 
of difference exists, but the gap is very slightly smaller. Thus, the 
direction of change is, as In Cincinnati, opposite to that predicted by 
the hypothesis. The size of the change, however. Is negligible. Inspec- 
tion of the month-by-month, data indicates an erratic pattern. On balance, 
the conclusion is that there is no effect of the treatment on attendance 
rates in Jacksonville. 

TABLE I 11-31 
ABSENTEEISM DATA—JACKSONVILLE 



MONTH (S) 



November 

December 

January 

February 

March 

April 



ABSENTEEISM 



EXP 



CON 



4.67 
6.80 
8.18 
7.75 
5.43 
4.84 



5.69 
7.75 
7.76 
8.61 
5.92 
5.73 



DIFF 



(EXP-CON) 



-1.02 
-0.95 
+0.42 
-0.86 
-0.49 
-0.89 



t-val. 



-1.87 
-1.16 
+0.66 
-1.12 
-0.68 
-1.47 



DIFF IN 
FAVOR 



EXP 
EXP 
CON 
EXP 
EXP 
EXP 



Fall Term 
Spring Term 



5.74 
5.14 



6.72 
5.83 



-0.98 
-0.69 



-1.67 
-1.13 



EXP 
EXP 
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The results for Oakland are presented in Table 111*32* In Oakland, 
the pattern is not entirely clear, but the general trend is in the direc- 
tion predicted by the hypotheses* At both the fall tena and the spring 
term observations, the absence rate at the EXP school is higher thar that 
at the CON school* However, the gap between the two schools is smaller 
in the spring than in the fall. This improvement in the level of absente- 
eism at the EXP school relative to that of the CON school is in the direc- 
tion predicted, even though the absenteeism at the EXP school remains 
slightly higher than that at the CON school. The impression gained from 
comparing the fall term index to the spring term index is confirmed by an 
examination of the month-by-month data, although this data also reveal 
that most of the effect is due to an unusually low absence rate at the 
EXP school during the month of March. 



TABLE III-32 
ABSENTEEISM DATA—OAKLAND 



MONTH (S) 


ABSENTEEISM (%) 


DIFF (%) 


t-val . 


DIFF IN 
FAVOR 


EXP 


CON 


(EXP-CON) 


November 


7.39 


5.32 


+2.07 


+2*74 


CON 


December 


8.44 


7.29 


+1.15 


+1.11 


CON 


January 


10.38 


7.39 


+2.99 


+3.06 


CON 


February 


8.76 


6.41 


+2.35 


+2.59 


CON 


March 


5.19 


6.36 


-1.17 


-2.11 


EXP 


April 


10.32 


8.19 


+2.13 


+1.89 


CON 


Fall Term 


7.92 


6.31 


-1.61 


+2.12 


! 

CON { 

! 


Spring Term 


7.76 
1 — 


7.28 


-0.48 


+0.67 


CON ! 

1 

1 

1 

k i 
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I The attendance patterns in San Antonio are suranarized in Table III-33. 

I The results from the fall term index and spring term index show a nearly 

I random pattern but one that is opposite to the direction predicted by the 

I hypothesis. That is, in the fall, absenteeism at the EXP school very 

j slightly lower than it is at the CON school. However, in the sprang, 

j absenteeism at the EXP school is very slightly higher than it U at the 

{ CON school. These differences are so small as to be negligible, but 

I they are in the direction contrary to the hypothesis. A study of the 

! detailed data tends to confirm the impression that the differences are 

essentially random in this case. Thus, the conclusion that there was no 

treatment impact on attendance in San Antonio. 



TABLE III-33 
ABSENTEEISM DATA— SAN ANTONIO 



1 

1 MONTH (S) 

i 

f 

j ^ 


ABSENTEEISM (%) 


DIFF (%) 


t-val. 


DIFF IN 
FAVOR 


EXP 


CON 


(EXP-CON) 


I November 


6.23 


5.98 


40.25 


+0.23 


CON 


December 


7.11 


7.69 


-0.58 


-2.67 


EXP 


January 


10.68 


10.62 


•H).06 


+0.04 


CON 


February 


8.38 


7.68 


+0.07 


+0.52 


CON 


j March 


7.83 


6.26 


+1.57 


+1.42 


CON 


j April 


6.09 


7.23 


-1.14 


-0.86 


EXP 


1 Fall Term 


6.67 


6.83 


-0.16 


-0.16 


EXP 


1 Spring Term 

: 1 


6.96 


6.75 


+0.21 


+0.19 

I 


CON 



J 

ERia 
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In general, the comparison of attendance rates at the four individual 
sites reveals a pattern of differences between EXP and CON schools that 
probably is best regarded as random* In the two sites which received 
the Teacher-Only model, Cincinnati and Jacksonville, there are no differ- 
ences large enough to mention* In Oakland, there is a small difference 
in the predicted direction. However, in San Antonio (which along with 
Oakland received the Parent-Teacher model) , there is a small difference 
opposite to the predicted direction. The general conclusion that must 
be drawn is that the application of the incentives models had no discernible 
impact on student attendance. 

Student Attendance Results — Model-Based 
Comparisons 

Becatise of the essentially nil evidence of any impact of the treat- 
ment on attendance rates in the individual sites, no attempt will be iriade 
to discuss or interpret the results obtained from the mod el -based 
comparisons of attendance rates* Such interpretations would be unwarranted, 
in light of the erratic individual site patterns* However, the results 
of such comparisons have been calculated, and these results are presented 
in Table III-3A, for the Teacher-Only sites combined; Table III-35, for 
the Parent-Teacher sites combined; Table III-36, for the EXP schools in 
the PT sites versus the EXP schools in the TO sites; and, finally. 
Table III-37, for the overall comparison of all EXP schools versus all 
CON schools* 

Student Attendance Results — Discussion 

There are several possible explanations for the general lack of 
impact of the incentives treatments on student attendance rates* Two 
explanations are substantive and one is procedural* One substantive possi- 
bility is that the introduction of the incentives treatment p^x» ce did 
not encourage a student to attend more regularly during the year* School 
staff members might knowingly ignore students who were chronic absentees, 
because, in their absence, there would be more time available for working 
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TABLE III-34 



ABSENTEEISM DATA~E/C TEACHER ONLY 





ABSENTEEISM (%) 


DIFF (Z) 


t-val. 


DIFF IN 
FAVOR 


EXP 


CON 


(EXP-CON) 


November 


5.50 


6.34 


-0.84 


-1.53 


EXP 


December 


6.86 


8.71 


-1.85 


-2.73 


EXP 


January 


9.13 


8.50 


40.63 


40.88 


CON 


February 


8.28 


9.00 


-0.72 


-1.05 


EXP 


March 


6.69 


7.15 


-0.46 


-0.60 


EXP 


April 


6.57 


6.48 


40.09 


40.12 


CON 


Fall Term 


6.18 


7.52 


-1.34 


-2.52 


EXP 1 


Spring Term 


6.63 


6.81 


-0.18 


-0.26 


EXP i 



TABLE III-35 



ABSENTEEISM DATA— E/C PARENT TEACHER 



i 

i 


MONTH (S) 


ABSENTEEISM (%) 


DIFF (%) 




DIFF IN 


: 
1 

1 


EXP 


CON 


(EXP-CON) 


t.val. 


FAVOR j 


i 
f 
I 

j 
* 

! 


November 
December 
January 
February 


6.83 
7.80 
10.53 
8.58 


5.55 
7.43 
8.53 
6.85 


+1.28 
4-3.70 
4-2.00 
4-1.73 


4-2.02 
4-0.47 
4-2.31 
4-2.29 


CON : 
CON i 
CON ' 
CON 


i 


March 


6.47 


6.33 


40.14 


40.24 


CON 




April 


8.27 


7.86 


40.41 


40.45 


CON 


Fall Term 
Spring Term 


7.31 

7.37 1 
1 


6.50 
7.10 


40.81 
40.27 


4-1.33 
40.45 


i 

CON 1 
CON 1 



Ill-lOO 



TABLE III-36 

ABSENTEEISM DATA— PARENT TEACHER-E/TEACHER ONLY-E 



MONTH(S) 


ABSENTEEISM (Z) 


DIFF (%) 




DIFF IN 


EXP 


CON 


(EXP-CWI) 


t-val. 


FAVOR 


November 


6.83 


5.50 - 


+1.33 


+2.14 


CON 


December 


7.80 


6.86 


-H).94 


+1.26 


CON 


January 


10.53 


9.13 


+0.14 


+1.58 


CON 


February 


8.58 


8.28 


+0.30 


+0.40 


CON 


March 


6.47 


6.69 


-0.22 


-0.32 


EXP 


April 


8.27 


6.57 


+1.70 


+1.88 


CON 


Fall Tens 


7.31 


6.18 


+1.13 


+1.87 


com 


Spring Term 


7.37 


6.63 


+0.74 


+1.05 


CON 



TABLE III-37 
ABSENTEEISM DATA~E/C ALL SITES 



M(»tIH(S) 


ABSENTEEISM (Z) 


DIFF (%) 




1 

DIFF IN 


EXP 


COti 


(EXP-CON) 


t-val. 


FAVOR 


November 


j6;66p 


5.97 


+0.09 


+0.22 


CON 


December 


7.26 


8.11 


-0.85 


-1.66 


EXP 


January 


9,72 


8.51 


+1.21 


+2.17 


COR 


February 


8.41 


7.98 


-0.43 


40.83 


CON 


t^rch 


6.59 


6.76 


-0.17 


-0.33 


EXP 


April 


7.31 


7.14 


+0.17 


+0.29 


CON 


Fall Term 


6.66 


7.04 


-0.38 


-0.93 


EXP 


iSpring Term 


6.95 


6.95 


+0.00 


+0.00 


j 
! 
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%i±th the students who are present. This, then, is one possible explanation 
as to why there would be no effort prompted by the incentive treatment to 
influence the attendance of a student who is chronically absent. On the 
other hand, for the student who is absent only occasionally, the pattern 
and possible causes are unpredictable, and hence there probably is little 
that the school staff can do to improve his attendance. 

This brings up the second substantive explanation. The percentages 
of absence indicated in this data are considerably lower than those rates 
thought to exist in similar low-income schools. This could itself indi- 
cate a variety of processes at work. Both the EXP and CON schools have 
very good attendance throughout the year and the small amount of absenteeism 
that does occur in these schools is probably due to real illness and other 
causes which have nothing to do with school-induced motivation or other 
pressure. So, there would be a ceiling effect operative on attendance. 
No treatment could improve the attendance much in view of its already high 
level. 

The third explanation is that the CON school faculties and principals 
worked harder at reducing absenteeism than they would have normally. They 
were not naive subjects in this project; and the impact of their control 
status, daily reinforced by the TAG and MAC presence, might have been 
sufficient to focus their attention on reducing absenteeism. This factor, 
then, would make the CON absenteeism rate a difficult benchmark against 
which to measure the net impact of offering incentives in the EXP school. 

Student Classroom Observation Results — Introduction 

The data to be discussed in this section were obtained from struc- 
tured classroom observation, conducted by workers trained and supervised 
by TAC. The observations took place in December (Time 1) and Hay (Time 2) . 
At each of these time periods, an observer spent a total of approximately 
80 minutes observing each class. The observations were made in four sei- 
sions of about 20 minutes each, in order to avoid any distortion that 
might be caused as a result of observing one atypical class session. The 



4 
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data were coded by means of a system devised for this project. The system 
included observatioris of behaviors of the teacher, behaviors of several 
students, and the focus of class activity* 

Briefly, the procedure used to record observation data was as 
follows. The observer began by choosing one student at randoQ as a sub- 
ject to be observed. For that student, the observer coded the kind of 
activity the student was engaged in during a ten-second period and the 
context of the activity. There were 14 different activity categories 
allowable and four context categories. These are presented in Table III-38. 
When ten seconds had elapsed, the observer chose another student at randora, 
and recorded that student's bdiavior for a ten-second interval. The ob- 
server continued in this fashion for three minutes, thus sampling the 
behavior of a number of students. After three minutes, the observer turned 
his attention to the teacher. Each ten seconds, the observer recorded 
the activity of the teacher (see Table III-38 for the possible codes 
for teacher activity), and the focus of that activity. Then, after three 
minutes of recording the teacher's behavior and its context, the observer 
again switched to recording the student behaviors.' The observation of a 
single session thus gave approximately six minutes of observation of the 
stt;dents, based on a number of students, and approximately six minutes of 
observation of the teacher, with approximately five unutes for getting 
set up and leaving. 

The data were recorded simply by counting the frequency of each 
activity and context separately for the teacher and student observations. 
For the analysis, the separate sessions within a single time point were 
combined, and these frequencies are the data presented in the tables that 
follow. Thus, the unit of observation is the classroom. However, in 
calculating the results shown in the tables here, no distinction was 
made as to differences between teacher, or differences within grade. 
Tha't is, all tables show totals by school or sets of schools. 
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TABLE III-38 

CLASSROOM OBSERVATION ACTIVITIES AND CONTEXTS: STUDENT 



IT»1 


ACTIVITY 


EXPLANATION 


HYPO- 
THESES 


1 


Reading 




E > C 


2 


Referring to Reading 


^uuucuu inaKes reierence to personax 
reading. 


E > C 


3 


Writing 




E > C 


4 


Reciting 


»7i.uucitu xo uexivenng prepareo ano/ 
or rote material. 




5 


Listening 




E < C 


6 


Being Tested 




E > C 


7 


Asking Academic Questions 




E > C 


8 


Answering Academic 
Questions 







9 


Criticizing 




E < C 


10 


Complimentitig 




E > C 


11 i 

1 
i 


Initiating 


Student <t!aT*t'C Hf^S^AMtk^ e* uOT*1r #«vi 

own 


E > C 


12 i 

" I 


Academic Game Playing 


Spelling bees, etc. 


E > C 


Nonproductive 


Goofing off, disrupting, etc. 


E < C 


14 i 

1 
1 

1 


Productive 


Behavior is not related to class but 






not disruptive. 


E > C 


Context' 








1 1 


Alone 




E > C 




Small Group 


2-5 students 


E > C 




Large Group 


6 students; 3/4 class 


E < C 




Whole Class 




E < C 
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Single-Site Comparisons — ^^Student Classroom 
Observation Results 

Observation Results > The data from each sltet.are-flrst presented 
In Tables III-39 through III-46. A formal test of significance has not 
been employed in the comparisons that follow, but rather patterns of 
change were searched for thorough comparisons of the t-value at Time 1 
with the corresponding t-value at Time 2. 

Cincinnati results are found in Tables III-39 and Ill-AO, the Time 
1 and Time 2 observations, respectively. In Cincinnati, only Item 13 
"Nonproductive", shows a clear change, witli there being markedly more of 
that student classroom behavior in the EXP school as compared to the CON 
school toward the end of the year (t-value - +3.24). In this case, the 
EXP scores remained about the s^e, whereas there was a significant de- 
crease in CON school nonproductive behavior from Time 1 to Time 2. This 
change in status in the area of classroom management was also observed 
more generally by tfAC. They reported that the EXP school's principal 
had been ill for extended periods during the year which had probably 
caused a deterioration in school discipline. On the other hand, the 
principal of the CON school had regarded the project competitively; per- 
haps this declina in nonproductive behavior was one consequence of a 
general tightening up in the CCX; school. This trend in Item 13 is somewhat 
confirmed by the decline of the EXP school with a concomitant Increase 
of the CM? school in Item 14, "Other Productive". The frequency of this 
behavior in the EXP school was reduced by one-half whereas it doubled in 
the CON school. Both of these trends are contrary to what was hypothe- 
sized. 

On the other hand, changes in Item 1, "Reading"; Item 6, "Being 
Tested"; and Item 7, "Asking Questions" were in the direction hypothesized. 
These three changes are very sli^t, involving only t-value changes from 
negative to positive from Time 1 to Time 2 with no significant difference 
in t-values in favor of the EXP school at Time 2. Only in "Being Tested" 
is there ai^ appreciable increase in the frequency in the EXP school from 
Time 1 to Time 2. The t-value changes in "Reading" and "Asking Questions" 
are due to decreases in these activities in the CON school* 
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Finally, the Cincinnati data indicate that there is a rather clear 
trend toward individual student attention in the EXP school classrooms. 
In the context, "One Student'*, the t-value went from -1.30 at Time 1 to 
+2.40 at Time 2. While the differences in the frequency of sma.ll and 
large group instruction remains about the same from Time 1 to Time 2, 
there is a relatively greater decline of "Whole Class" instruction in the 
EXP school. 

Tables III-41 and III-42 present the Time 1 and Time 2 Student Class- 
room Observation data for Jacksonville. There were relatively greater 
decreases of student behaviors in the EXP school in Item 7, "Asking Ques- 
tions", and in Item 14, "Productive". Both of these changes were not in 
the direction hypothesized. In five items there were relative changes in 
the direction hypothesized: Item 1, "Reading"; Item 3, "Writing"; 
Item 8, "Answering Questions"; Item 9, "Criticizing"; and Item 12, "Aca- 
demic Game Playing". In "Reading" the change is most pronounced with 
the t-values changing from -1.83 at Time 1 to +3.55 at Time 2. As can 
be seen from the means for Item 1, the frequency of this activity nearly 
doubled in the EXP school while it was being reduced in the CON school. 
The relative changes in these other four items are due either to marginal 
increases in the EXP school or decreases in the CON school and do not 
reveal any clear pattern. 

The contexts of instruction in Jacksonville changed opposite to the 
direction hypothesized in two areas, "One Student" and "Large Group". 
In the "One Student" context, the frequency of this mode of instruction 
in the EXP school was reduced by one-half from. Time 1 to Time 2 (t-values 
of +2.57 and -0.96, respectively). At the same time, "Large Group" 
instruction doubled in the EXP school from a Time 1 mean of 5.40 to a 
Time 2 mean of 11.08. Although there was an increase in "Large Group" 
instruction in the CON school, it did not approach the magnitude of the 
EXP school increase. Finally, it is worth noting that the frequency of 
"Whole Class" instruction increased somewhat in the EXP school; and although 
the t-values are small, they did change in the direction opposite to what 
was hypothesized. Therefore, the pattern in the Jacksonville EXP school 
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appears to be toward instruction in larger groups of students. This 
pattern is advanced tentatively, for it is in direct contradiction to the 
observations of the MAC monitor onsite. (ETS, 1972a, p, 43) 

Tables III-43 and III-44 present the Time 1 and Time 2 data for Oak- 
land* Among the 14 student activity items, there v^ere six relatively 
slight changes, all of which were in the direction opposite to what had 
been hypothesized* In Item 3, "Writing"; Item 5, "Listening"; Item 7, 
"Asking Questions"; Item 8, "Answering Questions"; and Item 11, "Initiat- 
ing", the EXP school declined relative to the CON school, with the Time 1 
EXP superiority either being reduced somewhat or with the CON school having 
a greater observed frequency at Time 2. Although none of the t- values is 
significant, it appears from an examination of the means for both schools 
at Times 1 and 2 that the major portion of the relative changes in these 
activities can be attributed to changes in the CON school. That is, the 
EXP school student activities are quite stable over time. 

The stability of student activities in the EXP school is reflected 
in the results of the context of instruction. Relative to the CON school, 
the pattern of the EXP school's context of instruction does not change 
over time. Its initial superiority in the direction of individualized 
instruction, reflected by a range of t-values from -3.34 to +4.01 as one 
goes from "Whole Class" to "One Student", is nearly replicated in the 
range and magnitudes of the Time 2 t-values. However, the appearance of 
EXP school stability is a bit misleading. From an examination of the 
means for both schools, it is interesting to note that the EXP school^s 
relative position was maintained while both schools moved toward indivi- 
dualized instruction. 

The Student Observation data for San Antonio are presented in Tables 
III-45 and III-46. There were five changes among the 14 student activity 
items. In Item 2, "Referring to Reading" and Item 3, "Writing", the 
changes were relatively large and not in the direction hypothesized. 
For Item 2 an examination of the Times 1 and 2 means reveals nothing; but 
for Item 3 it can be seen that although the frequency of "Writing" re- 
mained stable in the EXP school, it nearly doubled in the CON school. In 

(Text resumes on page III-116) 
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Item 1, "Reading"; Item 12, "Academic Game Playing"; and Item 13, "Nohpro- 
ductive"i the status of the EXP school improved relative to the CON 
school. Again, because the t-values at Time 2 are hot large, there is no 
instance of a clear EXP school superiority; however, the trend of tJie 
t-value changes from Time 1 to Time 2 is slightly favorable to the EXP 
school. An exaifilnation of the means reveals little interest except for 
Item 1, "Reading"; there it can be seen that the EXP school nearly 
doubled its efforts from Time 1 to Time 2, whereas the CON school fre- 
quency was quite similar over the two observations. 

The one change in the context of instruction in San Antonio was 
away from the direction hypothesized. The status of the EXP school 
relative to the CON school declined in "Whole Class", the t-values chang- 
ing from -2.12 at Time 1 to 40.61. at Time 2. That is, while both schools 1 
increased the frequency of this context of instruction, the EXP school 
increase was greater i with a Time I mean of 14.11 and a Time 2 mean, of 
18.89. 

Model-Based Comparis6ns--Studeht Classroom 
Observation. Results 

The first of the model-based comparisons to be examined Is; that for 
the Teacher-Ohly model, this comparison involves pboiing the data from 
Cincinnati and Jacksonville. The results for diese comparisons are 
presented in Tables III-47, and III-48. The results for the TO sites 
taken together resemble those for Jacksonville fairly closely . There are 
relative increases among Hie EXP students in the following activities: ^ 
"Reading", 'fetihg", "Answering Questions", and "Playing Games". There 
are decreases among the EXP students (relative to the CON students) in the 
activities of "Refer to Reading", "Criticizing", and "Other Productive". 
The focus indexes show a shift away from a focus on "One Student" and a 
shift toward. a focus on the "Large Group". Thus, these results provide 
seven confifmatiohs of the predictions made: "Reading"V "Writing", 
"Answering Questions" and "Referring to Reading" all increase; and "Playing 
Games", "Criticizing", and "Other Productive" decrease. There is one 
reversal of prediction, where there is a shift of the instructional focus 
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toward large group instruction. In general, no conclusions should be 
drawn from these data for the TO model because it would appear tiiat the 
Jacksonville results: washed out the trends iri: Cincinnati. 

The next of the model-based comparisons is that for the Parent- 
Teacher sites, Oakland and San Antonio. The results from this set of 
comparisons are presented in tables III-49 and III-50i These data indi- 
cate that the San Antonio results dominate the Oakland results . Thus, 
the activities of "Reading-^ "Listening"^ and "Criticizing" decreases, 
for confirmations of hypotheses. However, "Playing Games" increases; the 
activities of "Writing",: "Refer to Reading", "Asking Questions" j 
"Answering Questions", and "Initiating" all decrease^ which Is contrary 
to what was hypothesized. There appears to be ho significant trend to 
the context data. If anything^> it would appear that the EXP schools 
tend not to use large, group instruction quite as frequently as the CON 
schools i but eyeh here die former's Time 1 advantage deteriorates 
(t-value f r6m^3v5l to -1.86) . 

The third mpdel-based comparison is that for the EXjP students only 
in tihe PT si tes versus t^ EXP studoits only in the TO sitto. As men- 
tioned previously, this: comparison is subject to distortiph from differ-^ 
ehces between sites in the kinds of b^aviof that is typical. However, 
it does give some hints as to possible differences in behavior promoted 
by the two models. The results are presented iri fables 111-51 arid III-52. 
In die Parent-Teacher Expefim there is a relative increase 

iri t^. f ollwing activitie "Recitiriig", "Listening", "Asking Questions", 
"Playing Games", and "Nonproductive Bdiavior". There are decreases ins 
"Refer to Readirig", "Asking Questions", "Criticizing", and "Other Produc- 
tive Behavior". This represents a confirmation of five predictions. 
However, there are also four reversals of prediction: "Nonproductive" 
increases; and "Referring to Reading", "Asking Questions" arid "Other 
Productive Behavior" decrease. The context' indexes for the PT-E versus 
T6-E comparisons iridicate a relative increase in focus on "One Student" 
and on -Whole Class", and a relative decrease in focus on "Small Group" 
and "Large Group". This pattern could be interpreted in a variety of ways 

(Text resumes on page- IIT'^124) 
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The last of the model-based cdmparisoiis for the student observation 
data is the composite of all students in the ESP schools versus all stu- 
dents in the CON schools • These results are presented in Tables III-53 
and III-54. The comparison shows a relative increase for the EXP students 
in three activities. These are: "Read in ; , "Listening", and "Academic 
Gaming". There are relative decreases in three other activities; "Refer 
to Reading", "Asking Questions", "Criticising" ^ and "Other Productive 
Behavior". This represents a confirmation of three predictions: "Reading" 
increases; "Academic Gaming" increases; and "Criticizing" decrea&\es. 

There are three changes opposite to the direction predicted: "Referring to 
Reading", "Asking Questions", and "Other Productive Behavior" decreases 
The focus indexes for this overall comparison show no change from the first 
to second time point, with the EXP students receiving more individualized 
instruction relative to the CON students. 
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Student Classroom dbservatlon Results^ 
Discussion 

A summary of the results of the analysis of the Student Classroom 
data that would contribute either to a better understanding of the impact 
of incentives or to any organized body of knowledge about teaching is 
impossible. Table III-55 suimaarizes the results of the hypotheses, by 
site* There were a total of 72 opportunities to test the hypotheses con- 
cerning the 14 student activities and four context variables. In only 28 
out of these 72 cells was any change noticed. Of this 37% in which change 
did occur, only 13, or 46%, were in the direction hypothesized. Overall, 
the results are at best unclear. 

One generalization tihat does seem warranted, however, is that the 
activity of "Reading'' increases dilferentially in the EXP schools as com- 
pared with the CON schools. The pattern is reflected by an increase in 
trvailues greater than 1.00 in three of die sites, and an increase of 0.97 
in Oakland. However, the source of these increases, in terms of; the actual 
frequencies of the activity "Reading" in each of the schbpls at eadi of 
the times, is not is o simple as idight be thought. For instance, in Cincin- 
nati, the "Reading" results are produced through tihe occurrence of a slight 
increase in the: frequency of "Reading" at the EXP school , and a larger 
decrease in frequency of the same activity at the CON school. Likewise, 
in Jacksonville, the HJP school shows a considerable incre£ise in frequency 
of "Reading", and the CON school shows a sizable decrease. Thus, in both 
of these cases^ the apparent magnitude of the effect is determined partly 
by a change in the bebayior at the CON school over the interval between 
December and May ^ as well as by the changes in the behavior at die EJff 
school. 

For Oakland^ the pattern is somewhat different; There is an increase 
in the frequency of the ''Reading" activity at the EXP school, and no chang 
worth noting at the CON schools A similar pattern prevails at San Antonio^ 
where the frequency of "Reading" as an activity increases substantially at 
the E3ff school^ and shows little change at the CON schooli 
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im£ 111-55 

STUDENT CLASSROOM OBSERVATION RESULTS 
COMPARISON OF HYPOTHESES AND RESULTS 



ITEM/ 
CONTEXT 


ACTIVITY 


- 




RESULTS' 


I 




CIN 


JAX 


OAK 


SAN 


1 


Reading 


. E>C 


+ 




+ 


+ 


2 


Referring to Reading 


E>C 




• 






3 


Writing 


E>;C 




+ 


- 


- 


4 


Recitlne 


V ^ P 










5 


Listening 


E<C 


- 








6 


Being Tested 


iE > C 


+ 








7 : 


Aslding Ques tions 


E> C 










8 


Answering Questions 


E>C 










9 


Criticizing 


E< c ; 










10 


Complimenting 


E>C 










11 


Initiating 


E> c ; 










12 ; 


Academic Games 


E< C 








+ 


13 i 


Nonproductive 


E< C 












Productive 


E>C 












One Student 




+ 










Small Group 














Large Grpup 
Whole Class 













Key; 



Results are. in the direction hypothesized. 
Results are not in the direction hypothesized. 
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Thus, the source of the apparently uniform results for "Reading" across 
all four sites is found to be different in the two sites which e3q)erienced 
the TO model from that in the two sites .which experienced the PT model. 
For this particular pattern, a number of substantive explanations are possi- 
ble. For instance, it might be argued that in the two Parent-Teacher sites^ 
some form of parent pressure was being exerted which led to the increased 
use of class time for reading. The difficulty with this possibility is that 
it cannot be directly evaluated with the data that has been collected in 
the project. The Parent data does not bear directly on this question, and 
in any case is not complete. Moreover, one further problem involved in at- 
tributing the observed differences to parent pressure is that in the two 
Parent-Teacher sites, most of the parents were not aware of the existence 
of a project, much less its exact provisions, until quite late in the 
year. Of course, this could still have affected the Time 2 observations, 
which weire made iri-May^ But, this explanation must remain as a plausible 
speculation • 

More generally,' fe^^ a nuiriber of problems in drawing inferences 

from the classroom bbserva tid student behavior. For example, an 
alternate explanation for the increase of class time spent in reading is 
simply that thisT was a direct effect of the of feririg of incentives to tlie 
teachers for th^ their students. That is, the 

incr&ase in reading could be explained simply as evidence tiiat the offer 
of incentives had brought about a change ia the prbportipn of classroom 
time devoted to different kinds of learning. Since reading now was a 
kind of learning for which incentives were being offiered, the amount of 
time spent on reading was increased;. This ea^lanation,. like thJ one men- 
tioned previously, is plausible but not testable with the data at hand; 
That is, the increase does not occur uniformly across the four sites, 
and is in fact partly due to a decrease in the frequency the 
CON sdhoois. 

With the exception of "Reading", there are no consistent patterns or 
trends^ in the data that bear directly on the impact of incentives . "Writ- 
ing" decreases in Oakland and San Antonio, but it increases in Jacksonville. 
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"Individuaiized Instructioh^increases in Cincirihati, but decreases in 
Jacksonville. "Nonproductive" behavior increases in Cincinnati and pakiand, 
but only in Cincinnati is there a concomitant decline in "Other Productive" 
behavior. Meanwhile, in San Antonio ^ "Nonproductive" behavior decreases, 
as predicted. Across the four sites, then, at least as indicated by this 
data, influences other than the offer of incentives appear to be working. 

Within the sites, there are anomaiies that appear not to be in accord 
with the conventional wisdom about teaching and learning. In Cincinnati, 
for example, "Reading", "Being Tested", and "Asking Questions" all confirm 
the hypotheses. One might take this to mean that there was a tightening 
lip in the EXP school, a vigorous focus upon the subject matter and the 
objectives of the project. This assumption is made at great risk, however, 
because there was a simultaneous increase in "Nonproductive" behavior and 
decrease in "Productive" behavior i In Jacksonville^ "Asking Questions", 
generaiiy attributed to opeimess in the classroom, declined while other 
indices of openness increased. Furthermore, if "Asking Questions" is an 
isolated deviation from a trend toward more interaction and spontaneity i^ 
the classfoom, how does one explain tiie tr from "One Student" 

toward "Large Group" instruction? 

To further cqmplicate matters^ there is some congruence between the 
individual site data discussed here and the MAC observations and reports 
(Education Turnkey %ste^^ Inc . , 1972a & b) . . Some hint of the relative in- 
fluence of the absence of .the EH? school's piriTicipal reported by MAC 
can be gleaned from this data. In Oakland, it appears^ that the erosion 
of the E3ff school principal's authority along with maintenance of the CON 
school principar's strength and direction are reflected in the relative 
aeteripration of the EXP school.. However, although MAC made much of the 
principal's authority, less was made of the fact that a significant perr 
centage of the E3ff school faculty simply refused to acknowledge the exis- 
tence of the project. They claimed ^ from time to time quite vehemently j 
that they were in the project under protest and would therefore 
different. The data confirm their promise. 
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In addition to the variety of possible substantive explanations, there 
are also several possible sources of methodological complications involved 
in this observation data. A brief list of these will indicate the kinds 
.of issues that are involved: 

6 The categories are ipsative, and so a change in one category 

frequency will tend to imply changes in the frequencies in other 
categories as well. 

o The instrument is of unknown general reliability. 

6 Differences between observers in terb'5 of the way particular 

events are coded could contribute to differences between sites, 
although not to differences within sites. 

6 Differences could emerge in the skill of the observers between 
the first and the second wave of obseryations, and these dif- 
ferences would likely lead to changes in the pattern of frequen- 
cies that were coded. 

6 The use of drily the school averages for analysis i while desifabie 
as a means of dampening out also may conceal 

considerable differences be^een individua teachers as to the 
kinds^ of interaction patterns observed . 

o The interaction patterns observed may in fact be highly variable 
from situation to situation^ and the: totals found may well be 
largely the result of random sampling of the situatilphal differ- 
ences. 

These potential sources of method artifacts were recogriizcd,^ cbuld 
not be entirely neutralized, given the circumstances of the project. 
Coupled with the complex pattern of results observed at each site, they 
make it necessary to avoid drawing inferences from this data as to the 
specific kinds: of substantive changes that might be expected from the 
implemehtatibn of an incentives model. The frequency of reading time in 
the E)GP schools relative to the CON schools has been noted, but the reader 
should keep in mind the currents and eddies in the data that contributed 
to this outcome. 
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Analysis of Teacher Attitudes and jBehaviof 

Introduction 

Information concerning teacher reactions to the incentives models, as 
reflected in their expressed attitudes and selected behaviors, was collected 
by TAG via three instruments: (1) a Teacher Questioimaire, (-2) a Teacher 
Interview, and (3) a Structured Classroom Observation Form; Each of these 
instruments was administered at two times during the year. 

The form for the Teadier Questionnaire was hot given clearance by 
off icials at 0MB and OE until February, 1972, and so was first administered 
at that time. The number of teachers who completed the Teacher Question- 
naire at the time it was first administered, denoted Time 1^ was 103. The 
number of teachers responding to the Time 2 Teacher Questiohhaire, adminis- 
tered in May ^ was 81. There were 151 teachers defined as eligible to fill 
out these Questionnaires, and so the response rate for the Time 1 Question- 
naire was 68.2%, and the response rate for the Tiine 2 Questionnaire was 
53.6%. 

The Teacher Interview data collection was carried^out at two times 
during the year. The first wave of interviews i/ith the tM^ was con- 
ducted during November and December, arid the second wave was conducted 
during May . At each of these two times ^ intefviws were successfully com- 
pleted with all 15i eligible teachers. 

the Structured Classroom Obs.ervatidn of teadiers was carried out 
concurrently with the similar pbservatioh of student behavior described 
in an earlier section. Tliese Structured Observations, took place at two 
times during the year, namely December and May. They made use of repeated 
sampling of class sessions at each of the two times so as to avoid the 
possible biases that might arise from carrying out observation during one 
session which might be atypical, the observations were recorded by means 
of 14 categories. The coding scheme and the^r^^ 
are discussed in the section following this one. 
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The data from the Teacher Questionnaires and from the Teacher Interviews 
are similar in that all of it concerns a teacher^s verbally expressed atti- 
tudes on topics which might well be influenced by experience with the incen- 
tives model. Accordingly^ the results from both of these instruments will 
^be reviewed and discussed together in this section. 

Before proceedings it should be pointed out that, as with the Student 
Questionnaire and Student Interview data, the fact that the first adminis- 
tration of. the Questionnaire occurred so late in the year creates a serious 
problem in comparing the results of the Questionnaire data with the results 

of the Interview data. In addition, the relatively incomplete number of 

- / 

responses obtained to the Teacher Questionnaire (which was distributed via 
a mail-back method), and the decrease in response rate between the two 
waves of the Questionnaire, introduce addiltiohal possible sources of bias 
-into the results > 

Index Construction 

The data obtained from the 28 items of the Teacher Questionnaire were 
summarized by means of 11 indexes. These indexes w^^ in a 

manner similar to that used with the stiid^ '^^t is^ the questions 

had been develope^d with particular hypothesized proc in mind, arid so 

the items could be combined into indexes. Those questions which dealt with 
a single topic were combined as components for an index. The retipbrises 
made by each teacher to those items were added together to provide the 
index score for that teacher on that ihdex^ The concepts around which 
the Teacher Iridexes were built included: attitudes toward paying monetary 
incentives to teachers and parents; attitudes toward the sdiool; attitudes 
toward die students; perceived quality of interpersonal relations among 
the teachers; and preferences and opiriions^ about various teachirig methods. The 
name of eadi index and the list of items which compose it is given in 
Table III-56i 

Similarly, the data obtained from the 19 iteins of the Teacher Interview 
were also summarized in a set of four indices. The interview was relatively 
brief, and the smaller ninnber of items precluded developm^^^ of a large 

III-133 



TABLE iri-56 

INDEXES CONSTRUCTED FROM THE TEACHER QUESTIONNAIRE 



1. Teacher Attitude toWard Students 
Question No. 

5 tiy students' motivation to learn is... (very -^oor) ...(very 
high) 

6 My students' academic ability is. . . (very limited) (very 
capable) 

7 My students behave. i. (very poorly) . ./(very well) 
My attitude toward my pupils is ... (strongly negative) • • . 
(strongly positive) 



26 



2. Teacher Attitude toward School 
Questrion No. 

4 My school. . . (could stand alot of improvement) • • • (could serve 

as a model to others) 
12 My feeling about transferring to another school is. . • (very 

favorable) ... (very opposed); 
16 My attitude toward my school is . . . (strongly negative) . . . 

(strongly positive) 

3. Teacher Attitude toward Incentives^ 
Question No. 

^ offer of incentives to a teadier based on the achievement 

of his of her students is. ..(unlikely to increase achievement) 
• • • (likely to increase achieved 
21 The o|fer of incentives to a teacher based on the achievement ^ 

of his or her students is . . . (not proper) . . . (proper) 

4i Teacher Attitude toward Incentives to Parents 
Question No; 

15" The offer of incentives to parents based on the achievement 

of their child's class is. . . (not proper). .i (proper) 

24 The offer of incentives to parents based on the achievement 

of their dhild's cUss is. (unlikely to increase achieve- 
ment) . ..(likely to increase achievement) 

5. Teacher Attitude toward Relaxed Discipline 
Question No. 

Letting children move a^ classroom and talk to 

each other... (prevents tearhing). . .(fosters learnihg) 
Strict discipline in the classroom is an ii^ortant part of 
a child's education. ..(strongly agree) . ..(strongly disagree) 
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-Continued 
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TABLE III-56 (Cont'd/) 



6. teacher Perception of Principal's Attitude toward Innovations 
Question Nbt 

10 If I want to use new teaching methods or materials, n^r 
jprlnclpai Is likely to* . . (oppose me) . . .(assist TnB) 

19 My principal's attitude toward new teaching methods and 
materials ls« • • (negative) • (positive) 

7« teacher Perception of Cooperation Among Faculty 
Question No; 

17 Faculty meetings at my school are* ••(counterproductive) • 
(productive) 

18 Efforts of the faculty at my school to assist one another are 
• • . (counterproductive) • • • (productive) 

8^ Adult Relations in the School 

Question N6^ 

This index is the sum of indexes 6 and. ?• 

9 • Teacher Attitude toward Individualized Instruction 
Question No • 

22 If a child is allowed to proceed at his own rate during a 
year's time,, he is likely to^ • • (learn less) • i • (learn more) 

28 Individualized and self-paced ii^ to be>.^ 

(very inef fee tiveX. . . (very effective) 

10 . Teacher Attitude toward' Parents 
Question No> 

11 The role most of my pupils' parents play in their children's 
education is. . . (of ^ no significance) ... (of great significance) 

20 My attitude toward my children's parents is. . . (strongly nega- 
tiye) ... (strongly positive) 

lli Teacher Attitude toward Peer Tutoring ■ 
Question No. 

23 My feelings about having faster pupils tuto^ slow pupils in my 
classroom are. i. (opposed) . . . (favorable) 

27 Having faster pupils tutor slower pupils is likely to be... 

(very ineffective) . . . (very effective) 
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nuinber of indexes. The names of the indexes constructed from the Teacher 
Interview, and the items composing each, are given in Table III-57. 



Single-Site Comp arisons— Teacher Questionnaire 
and Interview Results 

The first, of the individual sites to be examined is Cincinnati. The 
results for Cincinnati are presented in Table III-58. The pattern of 
results is generally that there are few major changes, but that the changes 
which do occur are in a negative direction. For instance, of the 11 indexes 
constructed from the Teacher Questionnaire, only two show a substantial 
shift in the t-values from Time i to time 2. These are; "Attitude toward 
Students" and "Attitude Toward Individualized Instruction". For botii of 
these indexes, the shifts are in the direction opposite to what had been 
predicted by the hypothesis. The t-value at Time 1 for the"Attitude 
toward Students" index is. +1.30, and the t-value on the same index at 
Time 2 is ^0.32. This shift is one that may be due to regression effects, 
and so there is: some ambiguity surrounding it. Similarly, on the index 
for "Attitude toward Individualized Instruction", the t-value changes from 
+2.51 to +1.46. This shift also could be attributed to a regression toward 
the mean. However, since these are the only changes in t-value larger than 
i.pO, it is clear that, there is no positive Impact of the incentives model 
at Cincinnati, ais far as teacher attitudes expressed on the Questionnaire 
are concerned. When it is considered that the incomplete response rate 
probably led to a differential exclusion of teachers who are unfavorable to 
the project, thiis conclusion becomes even stronger. 

The general trend of the results, though not the specific attitudes, 
are similar when the data from the Teadier Inte^^ examined. Again, 

only two of the indexes from the Teacher Interview show substantial change 
in the t-valiies, and both of these changes are in a direction opposite to 
that hypothesized. The two indexes which show change are: "Attitude 
toward Modern Techniques" and "Relations with Parents". Here, too, the 
changes observed are in a direction that might be predicted from the opera- 
tion of a regression effect. For instance, the t-value for "Attitude 
toward Modern Techniques" at Time 1 is +2.35, and at Time 2 the t-value 

(Text resumes on page III- 141) 
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TABLE III-57 
INDEXES CONSTRUCTED FROM THE TEACHER INTERVIEW 



1. Teacher Attitude toward Incentives Idea and Project 
Question No> 

1 Do you think the payment of incentives to teachers is a good 
thing ' 

2 Do you think that the payment of incentives to parents is a 
good thing 

18 What are your feelings toward the incentives project being 

tried in your schoolidistrict 

2. Teacher Attitude toward Modern Techniques 
Question No, 

4 Have you made ixse of any techniques to individualize the 
curriculum you use in the classroom 

5 How often in the last two months have you requested special 
materials (books, films, etc.) not normally supplied 

8- In a normal.school week do you use peer tutoring as a class- 

room teaching technique 

12 Do you feel that the most important thing in the classroom 
should be discipline 

3. teacher Relations with Parents 
Question No> 

10 What percent of your students ^parents have V you in 
the last two months 

11 Do you feel you get on well with the parents of the chil- 
dren you teach 

13 How do you feel about the ability of the parents of the 
children ybii teach to help their children with their home- 
work 

3. Teacher Dedication to Teaching 
Question No > 

6 Are you satisfied with the prof essipno 
an individual 

=7 In the last two weeks have you ever considered transferring : 

out of the school you are presently teaching in 
9 Do you enjoy your students as diildren 

14 How many hours during a normal week do you spend on lesson 
preparation out of school 
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is +0;55. Similarly, the t-value for the "Relations with Parents" index 
at Time 1 is +2.26, and at Time 2 is -0.77. It is worth noting, however, 
that the shift in "Relations with Parents" occurs primarily because of a 
large decrease in the average obtained by teachers in the EXP school from 
Time 1 to Time 2. The shift on the index of "Attitude toward Modern 
Techniques", on the other hand, is accomplished through an increase in the 
average among the teachers in the CON school. 

In the second of the sites, Jacksonville, there are three Teacher 
Questionnaire indexes which show some change. These are: "Attitude 
toward Students", "Attitude toward Relaxed Discipline*', and "Attitude 
toward Peer Tutoring". The results for Jacksonville are presented in table 

All of these changes are in the hypothesized direction, and none 
of them is likely to be due to regression effects. That is, in all three 
cases » the size of the gap increases from Time 1 to time 2, and the direc- 
tion is in the favor of the EHP sdiopl. While these differences seem to 
be realy and seem also to reflect a general tendency to move toward a 
warmer and more relaxed classroom style, it is noteworthy that none of the 
indexes referring tc attitudes toward the incentives project, cr adult 
relations in the school, showed any sizable shifts . 1 

In view of the pattern observed from the Teacher Questioimaire indexes 
in: Jacksonville, one-would think it highly probable that the index for 
"Attitude toward Modern Techniques" drawn from the Teacher Interview, also 
wr>uld show a shift in the predicted direction. However, this is not the 
case. The t-value for the Interview-based index of "Attitude toward Modern 
Techniques" at Time 1 is +1.42, and at Time 2 the corresponding t-value is 
+1.46. In fact, none of the Interview-based indexes shows any change of 
note for Jacksonville. This lack of confirmation is somewhat puzzling, but 
does not weaken the general conclusion based upon the Questionnaire results 
that there probably was an improvement in classroom warmth in Jacksonville 
at the EXP schools. 

Inspection of the actual means for the three Questionnaire-based 
indexes which showed a shift in the predicted direction between Time 1 and 
Time 2 does lead to some qualification of the results. For the "Attitude 

(Text resianes on page III-- 145) 
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toward Students'* and "Attitude toward Relaxed Discipline", most of the 
observed change arises from the decline in attitudes among the teachers in 
the CON school. Thus,, the effect, if any, is an indirect one iEor these 
two indexes. For the third index, "Attitude toward Peer Tutoring", the 
change in t-values is due primarily to a rise in the mean among the 
teachers at the EXP school. 

Turning now to an examination of the indexes among teachers in Oakland, 
it is found that very few changes occur, and that some of these arc them- 
selves inconsistent. The results for Oakland are presented in Table III-60. 
In Oakland, the only index drawn from the Teacher Questionnaire which shows 
a change worth noting is that for "Attitude toward incentives to Teachers". 
Ike t-value for this index at Time 1 is -0.4Z, and at Time 2 the corres- 
ponding t-value is -K).80. This, then, is a shift in the predicted: direc- 
tion, although hot a large one. Because this is the only one of the 
Questionnaire-based indexes which shows a definite shift, and even it 
changes only a iuodest amoimt, it seems fair to say that the incentives 
model did not create any large changes in teacher attitudes in Oakland. 

An examination of the indexes derived.from the Teacher Interview for 
the Oakland data shows that there are two shifts large enough to be noted. 
One of these is oh the index which reflects "Attitude toward Incentives" 
and the other is oh the index for "Relations with Parents". The surprising 
point about the first of these shifts, the one oh "Attitude toward Incentives' 
is that it is in the direction opposite to the corresponding index from 
the Teacher Questionnaire. That is, the Interview-based index shows a 
t-value of +1.73 at Time 1, and a t-value of only +0.36, at Time 2. This 
shift, then, is opposite in direction to that from the Questionnaire-based 
index, and also may be the result of regression toward the mean. Inspec- 
tion of the actual mean scores for each of these indexes at each time point 
indicates that the shift on the Questionnaire-based index is the result 
of changes in mean level in both the EXP school and the CON school at 
Oaklahd. The mean scores for the Interview-based index indicate that the 
shift on that index is mainly the result of a decline in the attitude among 
the teachers in the EXP school. This does not resolve the paradox, but 

(Text resmes on page HI'- 149) 
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does indicate that the differences are not entirely due to changes &iDong 
the EM? teachers* It should be remeriered that the difference in the tiiae 
of the first wave data collection for the two instruments, plus the fact 
that the response rate to the Teacher Questionnaire was not extremely 
high, inay also be contributing to the lack of agreement in the results. 

One final note on these results is that the shift over time that was 
observed between the EXP and CON schools with respect to the Interview- 
based index of '^Belat ions with Parents** is due to a mixture of a decline 
in the level of this index among the teachers in the CON school, and a 
lesser decline on the same index among the teachers in the EXP school. 
Thus, this index, like the others, does not provide clear evidence of a 
positive impact of the incentives model. Instead; it indicates that there 
may be some benefit of the incentives model in slowing the rate of decline 
in relations of teachers with parents. Coupled with the other failures of 
the two instruments to indicate any sizable changes, this lack of consis- 
tency and positive impact leads to a conclusion that the incentives model 
had no general and positive impact in Oakland, as far as the attitudes of 
the teachers are concerned. 

The last of the four individual sites to be considered is San Antonio, 
the results for San Antonio are presented in Table III-61. They show a 
somewhat more positive pattern than any of the preceding three. However, 
these results are not extremely strong. 

In San Antonio, two of the indexes from the Teacher Questionnaire 
show changes from Time 1 to Time 2 large enough to be noted. These are: 
''Attitude toward Students** and **Attitude toward Relaxed Discipline**, Both 
of these changes are in the predicted direction. The change on the index 
for **Attitude toward Students**, however, could well be due to regression 
effects. The t-value for this index at Time 1 is -1,14, and the corres- 
ponding t-value at Time 2 is -0,05, The pattern on the index for ''Attitude 
toward Relaxed Discipline" is less likely to be the result of a regression 
effect, since the t-values change from -1,05 and +0*70, It should also be 
noted that the shift in t-value for the index of **Attitude toward Peer 

(Text resumes on page III- 15 3) 
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Tutoring*' shows a change diat is just on the borderline of being large 
enough to note. This chatige^ in which the t-value moves from -1.62 at Time 
I9 to «-2.59 at Tixae 2, is in a negative direction and is not likely to be 
theresult of a regression effect. 

The results for the Teacher Interview indexes in San Antonio tend to 
confirm those from the Questionnaire«*based indexes. All four of the 
Intervlew«-bascd indexes show a change that is large enough to be noted. 
For three of the indexes, the changes are in the direction predicted by 
the hypotheses. The Interview*-based index; for "Attitude ta^rard Incentives" 
shows a shift from an initial t-valiie of -0.53 to a later t-value of -^.66. 
This change, while not very large, is in the direction predicted by the 
hypothesis. The index for "Attitude toward Modern Techniques" in San 
Antonio is the one \^4iich dianges in the direction opposite to the predica- 
tion made by the hypothesis. This index shows a t-value of -0.21 at 
Time 1, and a t-valuc of -3.29 at Time 2. This change is fairly large, 
and is unlikely to be due to any regression effects, since it indicates 
a clear divergence of the means of the two schools over time. 

The Interview-based indexes for "Relations with Parents" and for 
"Dedication to Teaching" also show dianges in the predicted direction. 
The change observed on the "Relations with Parents" index (from a t«*value 
/ of -1.82 to a trvalue of -0.75) might be due to regression toward the 
mean, but this explanation cannot be offered for the shift observed in the 
"Dedication to Teaching" index. For that index, the Time 1 t-value is -0.33 
and the Tine 2 t- value is +1.38. 

The results in San Antonio tend to suggest a situation in which the 
participants generally became more positive in* their actions over time, 
but did not adopt the kinds of teaching techniques usually thought of as 
being new and nodern. This impression is confirmed by an examination of 
the actual means for the various indexes. For "Attitude toward Peer 
Tutoring" and for "Attitude toward Individualized Instruction", there is 
a decline in the average score obtained in the EXP school from Time 1 to 
Time 2. Although these declines are not large enough to create substantial 



III-153 



changes in the t-values, they are consistent with the results from the 
Interview-based index of "Attitude toward Modem Techniques". 

The overall conclusions that can be drawn from these comparisons of 
Teacher Questionnaire indexes and Teacher Interview indexes at the four 
separate sites must be tentative. However, it would appear that the 
incentives model did not produce any positive impacts in Cincinnati, and 
niay indeed have produced a negative Impact. In light of the other known 
information about Cincinnati, this should perhaps be stated instead as 
reflecting the disorganization observed in the EXP school because of the 
prolonged absence of the principal, and the competitive striving by the 
CON school in Cincinnati to demonstrate that it could excel without being 
awarded any incentives. The impacts in Jacksonville, though scattered, 
seem generally to be in a positive direction. The impacts of the incen-- 
tives model in Oakland, on the other hand^ are inconsistent, but probably 
negative. In any event, there is no evidence in Oakland for the contention 
that the incentives model had a positive effect. Finally, the situation 
in San Antonio seems to be that there was a general beneficial impact of 
the treatment, but no increase in use of modem techniques. On the contrary, 
the results suggest that in the San Antonio EXP school, the use of modern 
techniques was lessened during the project. 

Model-Based Comparisons—Teacher Questionnaire 
and Interview Results 

In view of the generally erratic results from the individual site 
comparisons, the model-based comparisons will not be discussed in detail. 
To present such discussion would implicitly suggest that the combined 
results are directly interpre table. Given the lack of consistency between 
the two sites which experienced the same incentives model (a phenomenon 
also noted in the analysis of the achievement data, and the student 
attitude and behavior data), the model-based comparisons are of questionable 
meaning. However, for the benefit of the interested reader. Tables 
III-62 through III-65 are presented showing these results. 
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The comparisons for the two sites at which the Teacher*Only model was 
used (Cincinnati and Jacksonville) indicate a slight shift in favor of the 
EXP school in the Questionnaire-based indexes for '^Attitude toward Relaxed 
Discipline'' and "Attitude toward Peer Tutoring". These results are pre- 
sented in Table III-62. However, the index of "Attitude toward Modern 
Techniques" from the Teacher Interview data shows a shift in the direction 
contrar>' to the hypothesis, although that shift is just slightly too sniall 
to be noted* 

In the comparisons involving only the schccls at the Parent-Teacher 
model sites (Oakland and San Antonio), there is a mixed pattern of results* 
These results are presented in Table III-63# There are shifts in favor 
of the EXP schools on the Questionnaire-based indexes of "A.ttitude toward 
Incentives to Teachers" and "Attitude toward Relaxed Discipline". However, 
there are shifts in the direction opposite to that predicted on the 
Questionnaire-based indexes of "Perception of Cooperation among Faculty", 
"Attitude toward Individualized Instruction", and "Attitude toward Peer 
Tutoring". Consistent with the negative patterns just mentioned, the index 
for "Attitude toward Modem Techniques" based on the Teacher Interview 
also shows a shift, in the direction contrary to that hypothesized. Thus, 
it would appear that in the PT sites, there was some general tendency to 
decrease the use of modern techniques in the EXP schools, as compared to 
the level of their use in the CON schools. The results in the PT sites 
from the Teacher Interview also indicate a relative improvement in the index 
for "Relations with Parents". 

As mentioned in the earlier sections where similar comparisons were 
made, the PT-E versus TO-E comparison is one in which site-to-site differ- 
ences can be confounded with any actual treatment impacts, and so it must 
be interpreted with extra caution. However, it does provide some infornia- 
tion as to the possible level of the marginal impact of the Parent-Teacher 
model over and above the impact of the Teacher-Only model. The results 
of this comparison are given in Table III-64. Only two of the indexes 
from the Teacher Questionnaire show differences over time in this set of 
comparisons. They are the index for "Perception of the Principal's Attitude 

(Text resumes on page III^ 166) 
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toward Innovation** and the "Adult Relations** index. The latter of these 
is 9 as Indicated in Table III-SS, a composite index which has the index 
for **Perception of Principal's Attitude toward Innovation** as one of its 
components. Therefore, these two results actually are reflecting a 
single phenomenon, namely that the teachers at the experimental schools 
in the FT sites find their principal more receptive than do the teachers 
in the experimental schools at the TO sites. In view of the replacement 
of one of the principals in the experimental school in a TO site (Cincinnati) 
for reasons of ill health, this result is not surprising. The Teacher 
Interview indexes for the comparison of FT-E versus TO«-E indicate, as did 
the same Indexes in the preceding comparison of schools at the FT sites, 
that there is a relative decline in **Attitude toward Modem Techniques** 
in the Farent-Teacher experimental schools as compared with the Teacher- 
Only experimental schools. Also, this set of comparisons indicates that 
the FT-E schools shift to a relatively more favorable position than the 
TO-E schools as far as **Relations with Farents** are concerned. But it 
must be remembered that this is due primarily to a slower rate of decline 
in the EXF schools. 

The last of the model->based comparisons is the overall combination of 
all four experimental schools compared with all four control schools. These 
results are presented In Table 111*^65. From this set of comparisons, only 
one of the 11 indexes based on the Questionnaire shows a sizable shift. 
This is the index for ''Perception of Cooperation among Faculty** which 
shifts in the direction opposite to that predicted by the hypothesis. Of 
the four indexes from the Teacher Questionnaire, there is likewise only 
one which shows a shift large enough to be noted. This is the index for 
**Attitude toward Modern Techniques** which also shifts in the direction 
opposite the hypothesis. 



Teaclier Questionnaire and Interview^- Results — 
DiscuQsion 

The preceding pages have presented the results obtained from the 
Teacher Questionnaire and Teacher Interview data. As far as any confirma- 
tion of the hypothesized relationships is concerned, there is relatively 

(Text resumes on page HI-- 169) 
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little support in these data. The positive outcoines are few in nuiober and 
are scattered throughout the sites and indexes. For the individual sites, 
of 44 hypotheses concerned with Teacher Questionnaire indexes, only six 
hypotheses turn out to be notable and in the predicted direction* At the 
same time, there are two hypotheses from the Teacher Questionnaire for 
which the outcomes were large enough to be notable, but in a direction 
opposite to that predicted. Similarly, of 16 predictions made for the 
Teacher Interview indexes, there were only four positive confirmations, 
matched by four instances in which the results were in the direction 
opposite to expectations. 

Despite the sparseness of these results » and the occurrence of outcomes 
opposite to those hypothesized, there are some general tendencies that may 
well reflect lnqpacts of the Incentives model. These have been mentioned 
in connection with the individual sites, and will be briefly reviewed here. 

One tendency that is contrary to the direction predicted is the rela- 
tive decrease in favorable attitudes toward modem approaches to teaching 
that was observed in the EXP schools of Cincinnati and San Antonio. This 
outcome perhaps indicates that one reaction to the Introduction of the- 
Incentives model Is a return to the traditional approaches. It may be 
that teachers feel these "tried-and-true" approaches will be as effective 
as any for the purpose of producing measurable increases in Reading and 
Hath achievement. A second instance of a possible general pattern Is in 
the relative increases in the Parent-Teacher EXP schools in the relations 
between teachers and parents, reported by the teachers on the interview. 
This result may be due, of course, to causes other than the PT model. It 
is created, in part, not by an increase in favor of the EXP schools, but 
simply by a lesser rate of decline. Thus, it should be regarded as tenta- 
tive. 

TWo general points can be made about these results. First, the find- 
ings opposite in direction to the hypothesis provide an example of one 
limitation that is inherent in this stage of research. Because it is rela- 
tively easy to explain the shift toward less favorable attitudes to modem 
techniques after the data have been presented, a critic might argue that it 
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would have been more desirable to have anticipated this possibility at the 
time the data collection was being planned, and to have included additional 
efforts so that relevant information establishing the nature of this process 
would be available. 

The difficulty with this suggestion as a general strategy, of course, 
is that there are a large number of alternate explanations which can easily 
be proposed, and an even larger nuniber of possible data patterns which 
might be found. Thus, to atteii?)t to cover all of these in a work aimed 
at preliminary knowledge would not be possible. Rather, it is the function 
of a first study such as this to narrow down somewhat the possibilities, 
so that subsequent research on the topic can in turn be concentrated more 
effectively. 

The second general point is that these results, like those reported in 
earHer sections, may reflect largely or entirely the iii5)act of what James 
Coleman has called "random shocks" (Coleman, 1964). This is not to say 
that there are not systematic processes occurring, but that they are made 
less detectable by accidental fluctuations in the level of the outcome 
variables. These "random shocks" act much like static in a radio, and make 
the reception of the message being transmitted difficult or impossible. 
In this project, the message being transmitted is whatever is really the 
inqpact of the treatment at the various sites. An understanding of that 
message is lessened, however, by the presence of additional components. 
The message is also made less understandable by whatever systematic distor- 
tion may be introduced by the instruments—in this case, by the techniques 
of data collection employed. The amount of this distortion is not precisely 
known, and so an additional measure of uncertainty is present. 

The general remedy for a situation in which communication is faulty 
is to use v^atever other channels and media may be available. In the 
present study, this approach was adopted in the form of collecting a variety 
of data from several sources. The next section takes up the analysis of 
one of the additional sources of data, the Teacher Classroom Observation 
data. 
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Teacher Classroom Observation Results — 
Introduction 

Teacher Classroom Observations were conducted by the TAC field repre- 
sentatives at the same time and In the same manner as the Student Class- 
room Observations were made* Table III-66 presents the 14 teacher activi- 
ties and the four context categories, along with the direction of differ- 
ence hypothesized for each of the 18 variables. 

Single-Site Comparisons — Teacher Classroom 
Observation Results 

Tables III-67 through III-74 present the results for the four sites. 
As In the analysis of the Student Classroom Observation results, formal 
tests of significance have not been used In the analysis of the Teacher 
Classroom Observation results. Rather, the analysis consisted of a search 
for patterns of change by means of comparing the E/C t-value at Time 1 
with the corresponding t-value at Time 2. Although this method of analy- 
sis may be unappealing to readers who are Interested In short, simple 
answers. It Is believed that the method Is In keeping with the complex 
nature of the project. That Is, this was a field project using only a 
few sites spread across the country. No attempt was made In the design 
to randomize for even the known sources of variance. Furthermore, there 
were delays in the implementation of the project and varying levels of 
knowledge among the participants at the four sites about the project 
objectives. In short, to present a few simple results in the analysis 
would be to imply that simple questions had been asked. This most clearly 
was not the intent or the fact in this project. 

Tables III-67 and III-68 present the Teacher Observation results for 
Cincinnati. From a comparison of the Time 1 and Time 2 t-values, there 
were four teacher activity changes in favor of the EXP school as predicted. 
In Item 1, "Testing"; Item 6, "Praising"; Item 12, "Supervising", the 
t-value differences from Time 1 to Time 2 were all greater than +1.99. 
Less substantial changes in favor of the EXP school were found in Item 2, 
"Explaining" and Item 12, "Supervising" with t-value differences of +1.07 
and +1.99, respectively. 

(Text resmee on page III- 175) 
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TABLE III-66 
TEACHER CLASSROOM OBSERVATION 



CONTEXT/ 
ITEM 


EXPLANATION 


HYP 


1 


Testing 


E >C 


2 


Explaining subject 


E >C 


3 


Lecturing Subject 


E <C 


4 


Giving reward: Giving or promising extrinsic 

mntivatOT* 


E >C 


5 


Punishing: ' Giving or promising physical punish- 


E <C 


6 


xraising* Dcuaenc Denavior or academic work 


E >C 


7 


bricicizmg: student behavior or academic work 


E <C 


8 


ijistening. io subject matter discussion, pre- 
sentation 


E >C 


9 


Questioning: About subject matter 


E >C 


10 


Drilling: Rote, repetition of formulas, etc. 


E <C 


11 


Administering: Lunch tickets attendance, etc. 


E <C 


12 


Supervising: Of student-initiated work 


E >C 


13 


Peer Tutoring: Establishing, supervising, en- 
couraging 


E >C 


14 


Enriching: Additional, not-for-credit, materials 


E >C 


One Student 




E >C 


Small Group 


Two-five students 


E >C 


Large Group 


Six-3/4 class 


E <C 


Whole Class 
- — ^ ^ 




E <C 
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On the other hand, there were two teacher activity variables for 
which the change was opposite to the direction predicted* Although none 
of these changes is large, the EXP school was above the CON school in 
Item 5, "Punishing" and Item 7, "Criticizing" • 

Among the context categories, it appears that the EXP teachers did 
spend significantly more time with individual students toward the con-* 
elusion of the project. The t-value for this context changed from +A.03. 
This change is due to a substantial increase in the "One Student" context 
in the EXP school combined with a slight decrease in this instructional 
context in the CON school* 

Overall, the pattern of change is difficult to interpret because in 
some cases the differences are due primarily to EXP school changes and 
in other cases to CON school changes, with little change in the EXP 
school* For example, in Item 2, "Explaining", and Item 6, "Praising", 
the mean frequency of the CON school decreased a fair amount whereas 
the EXP school frequency was relatively constant from Time 1 to Time 2. 
Among the three activities which changed in the direction opposite to 
that hypothesized, two of the changes. Item 5, "Punishing", and Item 7, 
"Criticizing", were primarily due to decreases in the observed frequency 
at the CON school. Taken at face value, the tables would indicate that 
the EXF school teachers were leading a somewhat schizoid life in the 
classroom, i.e.^ combining the more "traditional" behaviors of testing, 
punishing and criticizing with the "innovations" of individualized instruc- 
tion, praising and supervising work initiated by the students. That 
persons, including teachers, can simultaneously govern their activities by 
several diverse theories of human behavior and motivation is not denied 
in this analysis. Indeed, evidence for it emerges in the results. 
However, because the apparent pattern at the EXP school depends also 
upon what happens In the CON school, this set of results should be regarded 
with some suspicion. In Cincinnati, the EXP school did increase instruc- 
tion with individual students, more than doubling the frequency of 
activity in this context from Time 1 to Time 2. That much is clear and is 
also supported by the Student Observation results* However, conclusions 
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beyond this are fraught with danger. Even online 1, "Testing", where the 
EXP school tripled its efforts from Time 1 to Tine 2, there is no support- 
ing evidence to be found in the Student Observation data. 

Tables III-69 and III- 70 present the Teacher Classroom Observation 
results for Jacksonville. There were nine changes in teacher activity, 
two of which were in the direction opposite to what was hypothesized. 
Although the absolute and relative changes were very slight. EXP school 
teachers less frequently enriched their classrooms and more frequently 
lectured, neither of which was predicted. Among the seven items in which 
the direction of change was as predicted, five of the changes were sub- 
stantial, involving t-value differences from Time 1 to Time 2 greater 
than +1.99. These substantial changes were found in Item 2, "Explaining"; 
Item 6, "Praising"; Item 8, "Listening"; Item 9, "Questioning"; and Item 
11, "Administrating". Smaller t-value differences from Time 1 to Time 
2 were found in Item 1, "Testing" and Item 7, "CriticlEJiig" with the 
differences being +1.35 and -1.01, respectively. 

An examination of the t-values for the four context categories reveals 
two changes. Consistent with the predicted change, the EXP school improved 
its status in the use of "Small Group" instruction, with the t-values going 
from -1.00 at Time 1 to +0.29 at Time 2. Contrary to this slight change, 
the t-value for "Large Group" instruction went from -0.17 to +3.87, a 
rather clear change not in the direction hypothesized. 

Superficially, the pattern of change in the Jacksonville classrooms 
appears to be rather clear. That is, eigjit changes were as hypothesized. 
However, in two of these variables, "Testing" and "Praising", the major 
source of the change was the CON school, which reduced the frequency of 
these activities from Time 1 to Time 2. In four activities, "Explaining", 
"Criticizing", "Listening", and "Administration", both the EXP and CON 
schools reduced the frequency of the activity; but the EXP school reduc- 
tion was less. In the remaining two of the eight, "Questioning" and 
"Small Group", an increase in the EXP school frequency was matched by 
a decrease in the CON school frequency, thereby somewhat distorting the 
magnitude of the change. Furthermore, these latter changes could well be 
due to regression effect. 



III-176 



H H 



CO 



O U 



a 8 



H 



CO o 
CO CO 



CO 
H 



CO 



CO 



(0 



CO 



IX 



CO 



IX 



H 



f^oo^cocMr^oooo — 1 r: r*. > v 



ooooo.ooo oo 



%^ CM CM o> 



o o o o o o o 



I + + I 



I + + 



I + + 



I I 



CM so ^ CO O 



soosrcncMooHcM 



I I 



H O 



? ? ? 



CM O O H H 
+ I I + + 



o o 
I I 



ooooooooooooooS SSSSSSSSSSo 



sr so H o> o 

CO CO so H 0\ 



CM 



lOHiOHONsOOHOOOO 
mOCOCMCMsrcOOCMON 



coiocooocMco<r sfsrcMso 



O 00 00 O CM 



otnsr<rcMcMoo<r 



CM CO CM o 



^srOHsrcOsO OCOCMCMCM 



sf m o> o 



H 00 



O O 



CM »ri CM 



<r o o o CO so m 



HH^I32222 e;:J2CMyCMCMCMCMCMCM 



sr ON o so o\ 



CM 



VOOVOCMCOSOCMSO OO CMOnCM OX^i? 

cMsrcooocMcosr ^<rsrto 



inosoootncM<rco«^ON 

^'OXCMSTHCO 
• • • • • 

CM O so r-l CM 



CM CM CM to 0\ H 



SO 1^ 
• • 

O 1^ 



<M 



O O 00 0\ ON 



ON 00 to H so 



00 



ON CO in H 



cMovHinooiosbso 



OOHCM<r sOCMCMCOOOHCMsO^ 



CM CO sr in SO 



00 ON i-t 



o H CM CO sr 



Ills 



^ fr* H Ir* H b< b^. 
h4 h-i hi 



H H H 



H H Z a 



MMHHHHOCO 



III-177 



ERIC 



so H O ^ CM o 



HvOr^O r^CMH<M0OCMH 



O H O O 

+ + + r 



<M O 



\0*^000^0\0000v 



I + + + + + 



00 



.•r VO CM IT) VO 



0000000<M0 



I + + + I 



<M<^00 O<MHr^v0C0HHCMvOO<M00 



<M H O 
+ + I 



CO 



H O 



CO 



»yOOOO oooooooooooooo 

^OOOO OOO'OOOOOOOOOOO 
HHHH HHHHHHHHHHHHHH 



O 

H <M 



cn 

O PQ 

o u 

H OH 



IX 



o CM o vor^cnHmo*^t^mmo\cMr^c2 
CM CO vo csi mmcsf^o*^*^oocMcnH*^oooo 

%^4ncoco4ncMHO\vooocn 



CM m CO o 



H CO 



H O CM CM 



m m o h o 



co*^*^oor^vo<'ooco4no 



O vD H 



OOCM*^vOHHCOOO 



CM 



OOOCOO^^^CMvO^'OOCO 



CM CO vO 



cn o 
cn cn 



cn 



CM 



CM CM CM CMCMCMCMCMCMCMCMCMCMCMCMCM CM 
HHH HHHHHHHHHHHHHH 
HHHHHHHHHHHHHH 



cn 



O vO 'OX CO 



HmmoNoomcMoo 



CM m vo 



*^cMC0H coO'Or^oo««^^*^ooooin*^ooo 



<M CO O 



cMco^'^'cocMinooo^r^cMc^ 



a\ o CM vo\ooincMioa^<'0\comoo 

U^HCMO OiOHOOOOOCMOHHOivO 



O 00 CM O 



H CM m 



CM 



00 



<^ O O 00 CM 0\ ^ 



HCMco^mvor^ooo 



H H 



<M CO 



:s: s s 

pc] U U 



CO 



Q <^ 



a O 1-1 



III-178 



Even in the areas that changed in the direction not predicted, the 
resiats are anblguous. In Item 14, "Enriching", a slight EXP school de- 
crease in this activity was magnified by a slight CON school increase. 
The apparent EXP school increase in "Large Group" instruction is really 
the result of the combinations of a small increase there and with a nearly 
one-half reduction in this context in the CON school. The only clear 
result, and one that was not predicted, is that the EXP school increase 
in lecturing is due to a doubling of its frequency of this activity, with 
little change in the CON school. Unfortunately, this clear change is not 
matched by an increase in EXP school student "Listening". 

The Teacher Classroom Observation results for Oakland are presented 
in Tables III-71 and III-72. Among the 14 teacher activities, there are 
discernible trends between the Time 1 and Time 2 t-values in seven of the 
variables. Five of these trends, however, are in the direction opposite 
to what was predicted. The Time 1 to Time 2 differences are substantial 
for Item 8, "Listening", for which. the t-value went from +1.72 to -0.36; 
Item 11, "Administrating", for which the t-value went from -1.39 to 
+1.19; and for Item 12, "Supervising", for which the t-value went from 
+1.72 to -0.58. For the remaining two teacher activities, the observed 
trends are not as substantial: Item 5, "Punishing", had a Time 1 to Time 
2 t-value difference of +1.01; and Item 13, "Peer Tutoring", had a Time 
1 to Time 2 t-value difference of -1.60. 

For the two activities in which the change was in the direction 
predicted, the results differed in magnitude. In Item 7, "Criticizing", 
there was a substantial t-value change from +1.89 at Time 1 to -0.58 at 
Time 2. However, in Item 14, "Enriching", the change was not as pronounced, 
the t-value changing from -0.52 to 40.55. 

There are three trends found in the context categories, one of which 
was not in the predicted direction. In the "Small Group" context, the 
t-value went from-fl.57 at Time Ito -1.03 at Time 2, an indication that 
the frequency of this type of Instruction decreased in the EXP school 
relative to the CON school. However, the frequency of "One Student" 
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instruction in the EXP school relative to the CON school did increase as 
predicted, the t-value going from 40.66 to +2.70. There was also a 
decline in "Whoxe Class" instruction in the EXP school, as predicted, 
although the trend is very slight, with the t-value going from -1.37 to 
-2.62. 

The Teacher Classroom Observation results for Oakland are a mixture 
of consistent and erratic patterns. The consistent patterns are first 
that a majority of the changes were in the direction not predicted, and 
second, that this pattern is similar to what was observed of student 
classroom activities. However, there appears to be little congruence 
between student and teacher activities. For exajn>le, Oakland EXP students 
spent relatively more time "Answering Questions", but there is no corres- 
ponding increase in EXP teachersVAsking Questions". Furthermore, the 
student increase in "Listening" is not matched by a teacher increase in 
"Lecturing". On the other hand, there are some similarities. For example, 
the relative increase of EXP students in "Nonproductive" behavior is 
matched by increases in EXP teacher "Punishing" and "Administering". 
-Thus, although negative results predominate in both the student and 
teacher classroom data in Oakland, there is no consistency between them. 

In addition, the underlying causes of the changes vary In Oakland. 
In only two instances, "Criticizing" and "Small Group" instruction, were 
the t-value changes due to changes in the frequency of the activity In one 
school only. In the eight remaining areas where there was a discernible 
trend, it was the product of the confclnation of changes in both the EXP 
and CON schools. When this coafcining of changes is noted, the iin)act of 
incentives can be estimated in terms of what they did to trends in the 
EXP school as con?)ared to trends at the CON school. Tlius, in the COS 
school the context trend during the year was toward "Whole Class" and 
away from "One Student". In the EXP school, the tendency toward "Whole 
Class" was attenuated while the tendency away from "One Student" was 
reversed. Reversals of the tendencies established by the CC»I school are 
also found in "Listening" and "Supervising", both of which decreased in the 
EXP school; and in "Enriching", which increased in the EXP school. The 
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trends in these areas 9 therefore, nay well be due to regression effects. 
Both the EXP and CON teacher activity In "Administrating" declined from 
Time 1 to Time 2, but the amount of decline was greater In the CON school » 
which magnified the apparent Impact In the EXP school. 

These vicissitudes In the data Inhibit any attenpt to paint a detailed 
picture of the Oakland EXP school. Except for the substantial relative 
difference In the "One Student" context, which Is not supported by the 
student observations 9 there Is no safe generalization to be made about 
the Intact of Incentives on teacher activity. In support of the feeling 
that other sources of variation are at work {e.g.^ the principal 9 his- 
torical factors 9 different cohorts) 9 it is Interesting to note that both 
the EXP and CON schools increased their frequency of "Testing" threefold 
from Time 1 to Time 2. This absolute and relative increase in the EXP 
school testing was ei^ected and found in Cincinnati and Jacksonville* 
That it was found in the Oakland CON school is contrary to what was ex- 
pected. However 9 the increase is not too surprising when one remenfcers 
that the State of California statewide adilevement tests were to be 
administered soon after the observations were coiq>leted in April. This 
example is perhaps a bit too tidy and appropriate 9 but it does Illustrate 
the problems of not understanding or accounting for many sources of 
variation in an evaluation of a project of this nature. 

^"fS)les ^riI^3TSd"III-74 present the San Antonio Teacher Classroom 
Observation results. Among the 14 teacher activities 9 trends worth 
noting are found in five items. The trends were not in the direction 
predicted in two of these activities. Among these two reversals 9 one 
of the trends was rather substantial; in Item 89 "Lecturing" 9 for which 
the t-value went from -1.84 to +1.29. A minor trend that was not pre- 
dicted was found in Item 1^ "Criticizing" 9 where the t-value went from 
-1.73 to 40.05. 

There were three trends in the direction hypothesized among the 
teacher activities with a substantial relative diange in only (me activity. 
In Item 89 "Listening" 9 the difference in t-values from Time 1 to Time 
2 was +2.71. Less substantial trends were found in Item 11 9 "Administrating" 
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with a t-value difference of -1.51; and "Peer Tutoring", with a t-value 
difference of +1.27. 

There were two trends in the context of instruction in San Antonio, 
both in the direction not predicted* A substantial relative change was 
found in the "Whole Class" context, in which the status of the EXP school 
relative to the CON school changed froia a Time 1 t-value of -1*89 to a 
Time 2 t-value of 40.61. This net difference of +2.50 means that the 
frequency of instruction to the total classroom increased in the EXP 
school relative to the CON school. The second trend, although smaller, 
was also not predicted. The frequency of "One Student" instruction 
decreases in the EXP school relative to the CON school, with a net t-value 
difference of -1.15. 

A cursory examination of the means for Time 1 and Time 2 for both 
schools indicates that the sources of relative change appear to vary 
randomly. That is, in some cases the source of the trend revealed by the 
t-values is a large change in the CON school; in other cases it is a large 
change in the EXP school; and in others it arises out of the combination 
of slight changes in both schools. This has been found in the analysis 
of the other sites and is no surprise. However, in San Antonio the 
analysis is further complicated by the total absence of any recorded tallies 
for: the EXP school. Time 1, "Rewarding", and Time 2, "Punishing" and 
"Testing"; and for the CON school. Time 2, "Rewarding". The results pre- 
sented here agree with the raw data submitted by the field representatives, 
but because there is no way of checking the accuracy of their work, these 
data are highly suspect. 

Trends from Time 1 to Time 2 established by the CON school were 
reversed in the EXP school in two instances, and in both of these cases, 
"Testing" and "Rewarding", the missing data are a sufficient reason to 
dismiss them. In the remaining four areas where the trends are substantial, 
they are due to differences in rates of change between the two schools. 
"Punishing" and "One Student" instruction decline in both schools, but 
the decline is greater in the EXP school. "Listening" increases in both 
schools, but more so in the EXP school. The trend in "Administrating" is 
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due to an increase in this activity in the CON school, while it remains 
relatively stable in the EXP school. "Whole Class" instruction, on 
the other hand, increases in the EXP school but is constant from Time 1 
to Time 2 in the CON school. 

In San Antonio, the only iiiq)act of incentives revealed by this data 
is the trend toward "Whole Class" instruction. This is confirmed by the 
Student Observation results and is also supported by the relative ten- 
dency away from "One Student" instruction. Reversals in trends found in 
the CON school, which in any event could be a regression artifact, cannot 
be trusted because of the quality of the data in those items. For the 
other trends found, the EXP school differs not in kind but only very 
slightly in degree from the CON school. 

Modei-^Based Comparisons — Teacher Classroom 

Observation Results r^' 

The results for the model-based conparisons are presented in Tables 
III-75 through III-82. 

Tables III-75 and III-76 present the Tirae 1 and Time 2 results for 
the Teacher-Only sites » Cincinnati and Jacksonville. There would appear 
to be substantial trends in the direction predicted in five areas: 
Item 1, "Testing"; Item 2, "Explaining"; Item 6, "Praising"; Item 8, 
"Listening"; and Item 9^ "Questionning". There would appear to be a 
change opposite to the direction hypothesized in "Large Group" instruc- 
tion. 

However 9 to conclude that these are the impacts of incentives on 
teacher classroom behavior would be to ignore the fact that the Jackson- 
ville impacts predominate in the results. Only for Items I9 2, and 6 
are the trends similar for the two Teacher-Only sites. The trends in 
Items 8, 9 9 and "Large Group" are due to trends in Jacksonville only. 
Generalizations based upon an N of two are tisually questionable with 
this type of exploratory data; but to generalize in this case about 
"Listening", "Questioning" , and "Large Group" instruction, which are pri- 
marily attributable to Jacksonville and which are highly arJ^iguous in that 
context, would be misleading. 
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Tables III-77 and III-78 present the Teacher Classroom Observation 
results for the Parent-Teacher sites, Oakland and San Antonio. There are 
four trends worthy of any attention in these results, and they are all 
negative. In Item 1, "Testing"; Item 3, Lecturing", Item 12, "Supervising''; 
and the "Small Group" context, the trend is in the direction opposite to 
what was predicted. No conclusions can be drawn from this pattern be- 
cause each trend is the result of the data from one site only. Thus, 
the apparent PT model trends toward less "Testing" and more "Lecturing" are 
not found in Oakland and are almost entirely the result of the San Antonio 
data. Similarly, the apparent trends toward "Supervising" and away from 
"Small Group" instruction can be traced only to Oakland. In fact, the 
results at the two sites, when considered and compared individually, are 
even more confusing. Thus, for the combined PT model data, one must con- 
clude that there is no pattern in the results. 

The next comparison to be considered is between the EXP schools in 
the Parent-Teacher model and the EXP schools in the Teacher-Only model. 
Tables III-79 and III-80 present the data for these schools. Although 
this comparison introduces at least regional and local uncontrolled 
sources of variance to complicate further attenqpts to discern patterns, 
it does provide a rough indication of the marginal differences be- 
tween the two models. There are eight changes in the t-values from Time 1 
to Time 2 that are greater than 2.00. Six of these changes are in the 
direction opposite to what was predicted: Item 6, "Praising", decreases 
in the PT model EXP schools relative to the TO model EXP schools; Item 
9, "Questioning", decreases; Item 13, "Peer Tutoring", decreases; "Small 
Group" instruction decreases; and "Whole Class" instruction increases. 
In support of the hypotheses, one finds Item 2, "Explaining" and Item 4, 
"Rewarding" both increasing in the PT model EXP schools relative to the 
TO model EXP schools; while "Large Group" instruction decreases. 

Based upon an examination of the means, the results in "Whole Class" 
and "Large Group" instruction, and Item 2, "Explaining" can probably be 
attributed more to regression effect than to any real impact because the 
models' means change in opposite directions. In the other five areas, 

(Text reemea on page HI" 195) 
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the trends appear to be the result of an intensification in the PT model 
of patterns found in the TO model. For exan^le, the TJCP schools in both 
models tend away from "Small Group" instruction, but the rate of 
change is greater in the PX models Again, it would appear that the net 
iii?>act on teacher classroom behavior of offering incentives to parents in 
addition to the teachers is quite sli^it and more a matter of degree than 
kind* This is intuitive by reason, given the relatively high amount 
of physical and psychic isolation of classrooms from parental and 
community influences* 

The final model-based comparison to be considered is a comparison 
of all of the EXP schools with all of the CON schools • Although the 
caveat about probable sources of random variation obtained in this com- 
parison as in the previous model-based analyses, the trends uncovered can 
be useful if they are regarded as being highly tentative* 

Tables III-81 and III-82 present the data for the four EXP schools 
and the four CON schools* Trends are found in five areas, three of 
which were in the direction hypothesized • In Item 2, "Explaining"; 
Item 6, "Praising*; and Item 8, "Listening", the t-value differences 
were greater than 2.00 and as predicted* However, contrary to what was 
predicted, the EXP teachers increased "Ucturing" and "Urge Group" in- 
struction* Overall, therefore, there are very few discernible changes 
in teacher behavior in the EXP schools* Furthermore, it would appear 
that where there are changes they conbine to" form a mixed pattern of 
traditional and innovative activities* 

Teacher Classroom Observation Results — 
Discussion 

If the line between art and science has not been crossed earlier 
in this report, ^.nd there is *ain)le evidence that it has, then an attempt 
to summarize the results of the Teacher Classroom Observations provides 
that opportunity* Table III-83 summarizes the status of the hypotheses 
for the four sites in this project* The purpose of this project was not 
to prove or disprove theories or assumptions about the iaqpact of incentives; 
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TABLE III-83 

TEACHER CLASSROCJM OBSERVATION RESULTS 
COMPARISON OF HYPOTHESES AND RESULTS 



CONTEXT/ 

ixm 



1 

2 
3 
4 
5 
6 
7 
8 
9 
10 
11 
12 
13 
14 



ACTIVITY 



Testing 

Explaining 

Lecttnring 

Rewarding 

Punishing 

Praising 

Criticizing 

Listening 

Questioning 

Drilling 

Adninistrating 
Supervising 
Peer Tutoring 
Enriching 
One Student 
Snail Group 
Large Group 
Whole Class 



HYP 



E > C 
E > C 
E < C 
E > C 
E < C 
E > C 
E < C 
E > C 
E > C 
E < C 
E < C 
E > C 
E > C 
E > C 
E > C 
E > C 
E < C 
E < C 



RESULTS 



CIN 



+ 
+ 



JAX 



+ 
+ 



+ 
+ 
+ 
+ 



OAK 



SAN 



+ 
+ 



KEY: 



+ • Results are in the direction hypothesized. 

- » Results are not in the direction hypothesized. 



i 
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consequently 9 this portion of the analysis was focused t^on revealing 
trends within the variables and then discerning patterns among them. Thus, 
the symbols used in Table III-83, do not represent the results of formal 
tests of significance. 

There were four opportunities to examine 18 hypotheses. Among the 
72 cells 9 some trend was indicated in 35. Of those 35, 20 of the trends 
were in the direction predicted. Thus, trends in support of the hypo- 
theses were found in slightly over half of the cases where any trend 
existed (20/35 « 60Z). There are many probable sotirces of error in the 
instrument and procedures that have been discussed in the Student Class- 
room Observation results, and those factors are acknowledged in these 
results as well. However, the depressingly low percentages of both the 
findings and confirmations must also reflect on what is knoim, or thought 
to be kncwn, in aay organized w^ about teaching behavior. 

Examining the variables across sites, one must conclude that there 
is very little evidence of consistent trends due to incentives. "Testing" 
increased in the two Teacher-Only sites, but an examination of the EXP 
and COK school means in both sites would indicate that regression effects 
are operative in both cases. In any event, no similar relative increase 
was found in Oakland. It would be hazardous, however, to contrast the 
two models in this activity and to generalize from this comparison. The 
San Antonio data are suspect; and in fact, there were very substantial 
increases in "Testing" In both schools in Oaklanc^ attributable, it is 
assumed, to the California testing program. 

The relative f requcn<gr of "Explaining" in Cincinnati and Jacksonville 
is the result of declines in this activity in both schools at both sites* 
However, the magnitude of the decline was less in both EXP schools. 
"Lecturing" increased in two sites, which was not expected; however, only 
in Jacksonville, where the frequency of the activity doubled, can this 
trend be vigorously defended* On the other hand, this trend is in apparent 
contradiction to the concomitant increases in "Explaining" and "Listening" 
in Jacksonville. 
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Items 4, 5, 6, and 7 are logically related In that '"Rewar ding" and 
"Praising" are the reverse of "Punishing" and "Criticizing" • Contrasting 
these two pairs of items yields no firm conclusions except that the 
teachers appear to use different motivational styles about equally* Thus, 
no clear instructional ideology emerges, even in those three sites that 
evidenced a trend toward individualized instruction (Cincinnati, Jackson- 
ville, and Oakland) • Considered together because they imply a comdtment 
to innovative teaching, the results of "Supervising", "Peer Tutoring", and 
"Enriching" would indicate a very sliglit tendency away from that style 
in three of the four sites « 

Across the sites, therefore, the results are ambiguous as far as 
teacher activities are concerned; Furthermore, there does not appear to 
be any tendency to conduct instruction in smaller or larger grot^s* In 
both Cincinnati and Oakland^ the appsurent focus upon "One Student" may 
be due to regression. The same artifact m^ be found in both the 
"Small Group" and "Large Group" results in Jacksonville. As with their 
classroom activities, the teachers appear to have only a very sli^t 
preference for individual instruction. H^^ever, regression effects are 
so probable in this trend that it must be advanced very cautiously. 

Within eadi of the sites or the two models, an examination of the 
results is equally unprofitable. In Cincinnati, the EXP teachers appear 
to cori>lne **Puni8hing" and "Criticizing" along with "Praising". In 
Jacksonville, the consistency of predicted behavior over ••Praising", 
••Criticizing*^, and ••Listenlng^^ is contrary to the decline in ••Enriching^* 
and the increase in ''Lecturing*'. The apparent consistency between the two 
sites in ••Testlng^^, "Explaining", and ••Praising^* are probably not due to 
the particular incentives model found there but rather are attributable 
to site anomalies. Thus, they can only be noted in this analysis. 

In Oakland, the EXP teachers are relatively more frequently ••Punishing** 
but they also decline in "Criticizing^^. It would appear also, that the 
peripherals, ••Supervising^^ and ••Peer Tutoring^^ decline somewhat. However, 
there is a counter trend in the relative increase in ••Enriching*^ and ••One 
Studeht^^ instruction. In San Antonio, the more traditional activities of 
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"Lecturing" and "Criticizing" increase, contrary to expectation^ but 
so do "Listening" and "Peer Tutoring", a rather innovative practice. As 
an appropriate illustration of the dangers of drawing conclusions about 
the i]if)act of the Parent-Teacher model upon the classroom, it should be 
noted that in those Items or contexts vhere tendencies were found in the 
two sites the trends are always reversed. Thus, "Criticizing" increases 
in Oakland, but declines in San Antonio; "Peer Tutoring" declines in 
Oakland but Increases in San Antonio; and so on. As with the Student 
Classroom Observation results, the absence of any consistent pattern of 
trends is a consistent pattern in these results. 
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Analysis of Parent Attitudes and Behavior 

Introduction 

The preceding sections of this chapter have discussed results obtained 
from the various instruments designed to measure the attitudes and behavior 
of students and teachers. The results to be reported in this section will 
deal with the third major target population, the parents* Since this 
project was one in which one of the treatments involved parents directly, 
there is a particular interest In the changes that might be observed in 
their attitudes or behavior as a result of their being made eligible for 
cash payments. However » even aside from the direct impact of the incen- 
tlves model in the xmo sites where parents could receive payment, there 
is also the related question as to the role pl^ed by parents when an 
educational innovation is introduced to their school* Thus, a second 
reason for interest in the possible ityact of the project on parents is 
to see whether there is a spillover effect on parent attitudes and 
bdiavior, even in the sites where only the teachers actually were eligible 
for cash payment. 

To tap the attitudes and behaviors of the parents, two instruments 
were used. The first of these was a Parent Questionnaire, designed to 
le coi^leted at two times during the year by all parents in the CON 
schools as well as tfie EXP schools in all four sites. Due to delays in 
obtaining forms clearance, these Questionnaires were not available for 
use until March. Thus, as with the Student Questionnaire and the Teacher 
Questionnaire, the first wave of Parent Quest:: onnalres was administered 
in March. The Questionnaires were given to each child at school, and the 
child was asked to take them home, and to ask his parents to complete them 
and mail them back in an envelope which was provided, directly to the TAC 
office in Washington. The second-wave Questionnaire to the parents was 
administered in a similar fashion. 

At the TAC office, the Parent Questionnaires were compiled as they 
came in, and were allowed to be Included in the analysis until early 
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in July, at which time the Questionnaires which had been received were 
keypunched for the analysis processing. If it is assumed that the number 
of eligible students is 4042 (which is the average of the number of Student 
Questionnaires received on the two waves) , then the response rate for the 
Parent Questionnaire is as follows. There were 1329 usable Parent Question- 
naires received at the TAC office from the first wave, for a response 
percentage of 32.9%. There were 801 usable Parent Questionnaires received 
at the second wave, for a response percentage of 19.8%. These low response 
rates limit the clarity of interpretations that can be made of the results 
from the Parent Questionnaire data. It is reasonable to suspect that the 
parents who were more favorably inclined toward the project would more 
frequently take the troiible to respond to the request that they fill out 
a questionnaire. Also, it is equally reasonable to suspect that the 
parents who did return the Questionnaire are in general more educated and 
affluent and academically supportive than those who did not. Consequently, 
the results from this analysis must be interpreted with caution. 

The second instrument used to collect information from the parents 
was a Parent Interview Schedule. Interviews were carried out with approxi- 
mately 200 parents throughout the project at two times during the year. 
The first of these interviews was conducted in December, and 206 interviews 
were completed. The second interview was conducted during May, and 189 
were coispleted. The sampling procedure used for choosing the parents for 
these interviews was the same used for choosing the students. That is, 
a random sample of approximately 25 students was chosen to be interviewed 
in each of the eight schools; then their parents were interviewed; and 
then the students were interviewed. 

The content of the two instruments used with the parents includes 
questions asking about attitudes and opinions, on the one hand, and also 
questions asking about the typical behavior of the parent, and, occasionally, 
the behavior of their child. Because these items actually are reports 
of behavior rather than objectively observed behavior, they have been kept 
together in building the various indexes, and will be treated together in 
the discussion. 
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Index Construction 

As in the previous instruments, the content of the Parent instruments 
was determined by identifying various attitudes and behaviors which might 
be expected to be influenced by the introduction of the incentives models . 
Consequently, the indexes were constructed by combining all questions which 
dealt with a single broad topic, and treating the responses to each of 
these questions as components of a single index. There was a close but 
not perfect parallelism between the indexes developed from the Parent 
Interview and several of the indexes developed from the Parent Question-- 
naire* Thus, these indexes should be regarded as reflecting very similar, 
although not quite identical, content. 

The Parent Questionnaire contained 38 numbered items. From this 
instrument, 12 indexes were constructed and used in the analysis. The 
names of these indexes, and the items conqposing each, are presented in 
Table III-84. 

Similarly, there were six indexes constructed from the 20 items of 
the Parent Interview. These index names, and the items composing each 
index, are presented in Table III-85. 

Single^^Site Comparisons — Parent Questionnaire 
and Interview Results 

The first site to be discussed is Cincinnati. The results for; Cin- 
cinnati are presented in Table III-86. They reflect in general a some- 
what negative change in the EXP school relative to the CON school between 
the first and second administrations of the instruments. From the 
Parent Questionnaire, only two indexes show a change large enough to note 
between Time 1 and Time 2. These are the index for "Adult Help for 
Student" and the index for "Parent-Teacher Contact". For both of these 
indexes, the direction of the change is negative; that is, contrary to 
what had been Iqrpothesized. For the index of "Adult Help for Student", 
the change is due to a decline in the level of adult help at the EXP 
school in Cincinnati, and a very slight increase in the level of adult 

(Text resumes on page 111^210) 
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TABLE. III-84 

INDEXES CONSTRUCTED FROM THE PARENT QUESTIONNAIRE 



1. Help for Student 
Question No> 

19 Your child is helped in math by someone else at home 

20 Your child is helped in math by an outside adult 

25 Your child is helped in reading by someone else at home 

26 Your child is helped in reading by an outside adult 

2. Parent-Teacher Contact 
Question No* 

34 Number of meetings with child^s teacher in. past two months 

35 Number of phone conversations with child ^s teacher in past two 
months 

36 Number of written notes from child's teacher in past two months 

3. Parent-Parent Contact 
Question N6> 

37 Number of meetings of parents attended at school in past two 
months 

38 Number of meetings of parents attended not at school in past 
two months 

4« Parent Attitude toward Child ^s Achievement 
Question No> 

17 How well is your child doing in math. (poorly) ... (well) 

18 How well is your child doing in reading. .. (poorly) ... (well) 

5. Parent Interest in Child's Achievement 
Question No. 

9 To me, my child's ability to do math well.. .(does not matter) 

...(is very important) 
10 To me, my child's ability to read well... (does not matter)... 

(is very important) 

6. Parent Attitude toward School 
Question No. 

1 My child's schools .. (could stand improvement) ... (could serve 

as a model to others) 
3 How do you feel about your child's school. .. (dissatisfied) .. • 

(satisfied) 

/ My feeling about discipline at ray child's school is. «. (dis- 

satisfied) . . . (satisfied) 

7. Parent Attitude toward Teacher 
Question No. 



2 My child's teacher is... (one of the worst in the school)... | 

(one of the best in the school) ! 

6 When it comes to my child's education, his teachers. .. (do not j 

cave) • • . (do care alot) | 

11 How do you feel about your child's teacher. .. (dissatisfied) .. . ! 

(satisfied) 



-Continued 
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TABLE III-84 (Cont^O 



7. Parent Attitude toward Teacher (ContM.) 
Question No, 

13 How is your child *s teacher doing in teaching arithmetic to him 
. . . (poor job) . . . (good job) 

14 How is your child^s teacher doing in teaching reading to him... 
(poor job)... (good job) 

8. Students Help Students 
Question Nov 

22 How often does your child get help with arithmetic from another 
child . . . (never) . . . (about every day) 

23 How often does your diild give help with arithmetic to another 
child. .. (never) ... (about every day) 

28 How often does your child get help with ref.ding from another 
child. . . (never) . . . (about every day) 

29 How often does your child give help in reading to another child 
... (never) .. .(about every day) 

9. Parent Attitude about Motivation 
Question No. 

5 A good way to get my child to do well in school, is to punish him 

when he does poorly ... (strongly agree) ... (disagree) 

15 A good way to get ny child to do well in school is to praise hiuj 
when he does well. . . (disagree) . . . (agree) 

10. Adults Help Neighbor's Children 
Question Ko. 

21 How often does an adult in your home help other people's child- 

ren with arithmetic; . . (never) . . . (about every day) 

27. How often does an adult in your home help other people's child- 

ren with reading. .. (never) ... (about every day) 

11. Parent Attitude toward Incentives to Teachers 
Question No> 

4 Offering teachers more money if their pupils learn more is... 

(not likely to work) ... (likely to work) 
12 Offering teachers more money if their pupils learn more is... 

(not proper) . . . (proper) 

12. Parent Attitude toward Incentives to Parents 
Question No. 

8 Encouraging parents with money if their child's class learns 

more is . . . (not proper) . . . (proper) 

16 Offering parents money if thfeir child's class learns more is 
. . . (not likely to work) . . . (likely to work) 
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TABLE III-85 
INDEXES CONSTRUCTED FROM THE PARENT INTERVIEW 



1. Adult Help for Student 
Question No. 

4 Do you help your child in doing his/her homework 

9 Do you get extra reading and/or arithmetic learning materials 

for your child 

15 Do any adults outside of your home help your child with his /her 

homework 

2. Parent-School Contact 
Question No* 

2 Do you attend school activities, e.g, PTA meetings 

12 Do you think parents need to be familiar with their child *s 
teacher 

13 Have you visited or met with your child *s teacher 

20 Have you discussed or attended meetings about the payments of 

money project with your child *s teacher or other sdiDol official 

3. Parent-Parent Contact 
Question No> 

5 Do you think that groups of parents working together can improve 
their children's school grades 

6 Have you organized any extra help sections for reading/arith- 
metic work for your children, with other parents in the neigh- 

• bprhood 

19 Have you discussed or been to meetings with other parents about 

the payments of money project 

4. Parent Satisfaction with Child's Achievement 
Question No> 

7 Are you satisfied with how your child is doing in math 

8 Are you satisfied with how your child is doing in reading 

5. Parent Supports Child's Learning 
Question No. 

3 How often do you encourage your child to do well at his school 
work 

4 Do you help your child in doing his/her homework 

9 Do you get extra reading and/or arithmetic learning material 
for your child 

10 Do you encourage your child to use the public library 

6* Parent Report of Student Learning Habits 
Question No* 

11 Kow often does your child use the public/school library 

14 Do your child's friends work with your child on their reading/ 

arithmetic homework 
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help at the CON school iu Cincinnati. On the other index, that for 
"Parent-Teacher Contact", the observed net shift is the result of a com- 
bination of a decline at the EXP school, and a larger rise of such contact 
at the CON school. Thus, the EXP school begins with an average score on 
this index at Time 1 of 2.43, and has declined to a score of 1.83 at 
the Time 2 administration. However, the CON school, which began at approxi- 
mately the same level, 2.49, rises to an average of 4.58 at the Time 2 
administration. This result seems once again to confirm that in Cincinnati, 
the CON school was making real efforts to improve and operate at the very 
best level possible. 

The indexes constructed from the Parent Intezyiew for Cincinnati con- 
firm the shift toward relatively less contact between the parents and the 
sdibdl staff in the KCP school. The definitions of these two ii^dexes 
are hot quite identical, and the index from the interview eiqihasizes con- 
tact not just with the teacher, ^^but .with other aspects of the school as 
well. Nqietheless, the two indexes should be closely coinpa^ An 
inspection of Table III-86 indicates that indeed there is a notable net 
shift on the Interview-based index, "Parent-School Contact", in the 
direction opposite to the hypothesis (buu consistent with the correspond- 
ing result from tiie Quest iomxaire-based index) . Further inspection of 
the means in the table indicates that this pattern is attributable to 
the mixture of a large drop in contact rate at the lEXP school and a 
smaller rise in contact rates at the CON school. Thus, both instruments 
suggest the same pattern. 

The Interview-based index for "Parent-Parent Contact" at the EXP 
school shows an increase relative to that of the CON school. The t-value 
at time 1 for this index is -l.l6, and at Time 2 the corresponding t-value 
is 40.53. This change occurs through a type of lesser deterioration 
process. That is, the l^vel of pari^t contact with other parents, as 
reflected in the Interview-based index, actually declines between Time 1 
and Time 2 from an average of 4.88 to an average of 4.50. However, ^the 
CON school suffers a greater decline, from a Time 1 average of 5.59 to 
a Time 2 average of 4.14. In other words, this apparently positive result 
in the EXP school is really the lesser of two evils. 
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A third index based on- the Parent Interview which shows a notable 
change from Time 1 to Time 2 is that for the "Parent Satisfaction with " 
Child's Achievement". This index j top, is not quite identical to the one 
on the Par ;int Questionnaire. The Interview-based index refers specifi- 
cally to "satisfaction", and the index based on the Questionnaire is 
phrased in terms of general perception of how well the child is doing in 
his school work. The t-value for diis index based on the Interview 
changes from +0.80 at Time U indicating that the parents in the EXP 
school are more satisfied, to a value of -0.53, indicating that at Time 2 ' 
the parents in the CON school are more satisfied. This change comes about, 
however, primarily through a rise from 6.77 to 7.68 in the average for the 
CON school. The EXP school is nearly constant at the two times; its 
Time 1 average being 7.44 and. its Time 2 average being 7.25. 

The changes observed in the t-values in Cincinnati are not of the 
sort that are likeiy to be due to regression toward the mean. Thus, it 
seems reasonable to draw the tentacrlve conclusion diat the impact of 'the 
incentives model on parent attitudes and behaviors in Cincinnati was not 
positive, and possibly was negative. Qf course, in interpreting these 
results as indicating a negative intact of the incentive treatment j one 
is ignoring the peculiar history of die project in Cincinnati. At that 
site, there was a prolonged absence of the EXP school principal, aid an 
egression of an unusually high motivation at the CON school. 

The second of the individual sites is Jacksorivillei The results of 
the Parent Questipmiaire and Parent Interview at Jacksonville are pre- 
sented in Table 111-87. They indicate a generally negative, although 
somewhat mixed, pattern. Of the 12 indexes constructed from the Parent 
Questionnaire, there are two -*rtiich show a notable shift in the predicted 
direction, and five which show a notable shift in the direction contrary 
to that predicted by the hypotiies is. Similarly, of the six indexes 
constructed from the Parent Interview, there is only one which shows a 
positive shift, and two which show a negative shift in the Jacksonville 
data. 
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Of the Questionnaire-based indexes, the negative shifts occur oh 
the following: "Parent Attitude toward Child's Achievement", "Parent 
Interest in Child's Achievement"^ ''Parent Attitude toward Teacher", 
"Students Help Students", and "Parent Attitude about Motivation". The 
observed shift in the t-values for "Parent Attitude toward Child's Achieve^ 
meht',' from a Time 1 t-value of +2.70 to a Time 2 t^vaiue of W.40, is 
caused by a rise in the average at the CON school from 4.84 to 5.05, and 
a small decline at the EXP school from 5.29 to 5.13. The shift is one 
that: could be due to a regression toward the mean. Similarly, the index 
"Parent Interest in Child's Achievement" shows a change from a Time 1 
t-value of +1.16 to a Time 2 t-value of -0.35. This change arises from 
the small variance of the index, which in all four cases has averages 
greater than 5.96, with a logical ceiling of 6.00. Thus, this result must 
be regarded as due probably to chance fluctuation and the very small 
observed variance on these items . 

, Similarly, the shifts observed on the other indexes are due to 
complex causes. For instance^ the shift on the index "Parent Attitude 
toward Teacher" from a t-value of +2.57 to a t-yalue of +0.93 is due almost 
entirely to a rise in the attitude held by parents of diildren in the CON 
school, from an average of 12.87 at Time 1^ to an average of 13.40 at Time 
2. The change in the "Students Help Students" index is due to shifts in 
both schools . The EXP school shifts down .and the CON School shifts up, 
thus making the net pattern on which the t-yaiue goes from +1.67 at Time 
1 to -0.25 at Time 2. The last of the Questiohhaire--bas_ed indexes whidi 
,sKow substantial shift in t-value is that for "Parent Attitude about / 
Motivation". On this index^ parents in both schools are increasing in 
positive attitude over time, but those in the CON sciidol show a greater 
increase . 

Turning now to the indexes derived from the Parent Intv rview in 
Jacksonville, one finds that there are shifts contrary to the predicted 
direction on the indexes "Pafeht-Pareht Contact" and "Parent Report of 
Student Learning Habits". The change on the "Parent-Parent Contact" 
index is one which might well be due to a regression effect. That is, the 
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t-valiie at Time 1 is— 3.00, and at Time 2 is +0.67* On this index, 
bdthi schools decline considerably from Time 1 to Time 2, but the EXP 
school declines more. The average at the EXP school on the "Parent-Parent 
Contact" index at Time i is 6*41 and at Time 2 the corresponding average 
has shruidc to 3.46. In the CON sdipol, the time 1 average is 5.05, and 
it shrinks to 3.24 at Time 2. 

The observed shift in the "Parent Report of Student Learning Biehavibr" 
index occurs through two cionulative changes: an increase over time at the 
CON school and a decrease over time at the EXP school. Finally, the 
positive change that was obsexrved on the Interview-based indexes 9 ccurs 
on the index for "Parent Satisfaction with Child's Achievement". On- this 
index, the t-value changes from -1.67 at Time 1 to +0.34 at Time 2. Thus, 
this could be diie to a; regression effect. 

Despite the varied sources of t^iese differences, it does seem fair 
to conclude that the inqpact of the incentives model on parent attitudes 
and bciiavior in Jacksonville was more negative than^ positive . 

The third individual site to be examined: is Oakland. The results 
for baJcliand are presented in table III-88. The^e results indicate a 
generally positive impact. There are four differences in the predicted 
direction on the Questionnaire-based 1^^^ and two differences in 

the direction opposite to what was predicted. * On the Interview-based 
indexes, there are four positive differences, and no difference contrary 
to prediction. Interestingly, thie pattern of results indicates a general 
improvonent of the parents-* attitudes about their child and the school,, 
but the two instances of a prediction contrary to hypothesis are both 
on the attitudes toward the payTnent of incentives . 

As can be seen from Table III-88, the index for "Parent-Parent Contact" 
has a t-value at Time 1 of -W.Slr and a t-value at Tiine 2 of +2.28. This 
increase is the result of a cdhsiderably greater increase in the rate of 
.such contact at the E3ff school, although there is also an increase at the 
CON school. The average on this index at the EXP school at Time 1 is 
5.68, and at Time 2 is 8.12. The average at the CON school at Time 1 is 
5.54, and at Time 2 has risen only to 6.92. 
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The "Parent Attitude toward Child's Achievement" index at Oakland 
shows a rise in t-value from -3.29 to -0.45. This shift could be due to 
a regression effect; however, it occurs entirely through a rise over time 
in the level of this index at the EXP school. That is, the average at the 
EXP school changes from 4.89 at Time 1 to 5.29 at Time 2, while the average 
of the CON school is unchanged at 5.38 for both times. Because two items 
made up this index i 6 was the logical high score and the CON school might 
have been constrained by this ceiling effect. 

The observed shift in the "Parent Attitude toward Teacher" index at, 
Oakland also occurs primarily through a rise in level at the EXP school. 
That is, the level for this index at the EJff school changes from 13.03 
at Time 1 to 13.66 at Time 2, while the level, at the CON school moves 
only from 13.56 to 13.68. 

\ \ 

The last of the Questionnaire^based Indexes in Oakland to show a 
positive effect is the index for "Parent Attitude about Motivation". The 
t-value on this index shifts from -1.16 at Time 1 to -K). 32 at Time 2. 
The shift is the result of a decline in the preference for praise as a 
motivational technique in both cfchools, but a greater .decline at the CON 
schboli 

The two indexes based on the Questionnaire which chow a differaice 
contrary to prediction in Oakland both deal with the payment of incentives. 
The index "Parent Attitude toward Incentives to Teachers" changes from a 
t-value of +3.80 at Time 1 to a value of +2.47 at Time 2. This shrinkage 
is due in part to the fact that although both schools do increase on this 
index,, the CON school ejdiibits a greater increase in f avorableness than 
does the EXP school. Oh the second index relating to incentives , "Parent 
Attitude toward Incentives to Parents", the pattern is similar, although 
less strong. On this index^ part of the explanation for the decline in 
the t-value is due to the difference in case bases between the Time 1 
data and the Time 2 data . It should be noted that the point biserial 
correlation for this comparison indicates that the effect is quite small, 
but in the positive direction over time. The positive direction is con- 
sistent with the fact that this is one of the sites at which the Parent- 
Teacher model was used. 
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The first of the indexes based oh the Parent Interview is that for 
"Adult Help for Students". This index shows a positive shift (from a 
t-valiie of -0.53 at time 1 to a t-^value of +1.95 at Time 2) . This shift 
is the result of a large increase in average at the EXP school (from 8.18 
to 10.20) and a slight increase at the CON school (from 8.62 to 8.92.). 
The index for "Parent-Parent Contact" based on the Interview also shows 
an increase in t-value. This increase also is due to a greater rise at 
the EXP school than at the CON school. The third Interview index for 
which a ^shif t large enough to note occurs in Oakland is "Parent Satisfac- 
tion with Child's Achievement". The t-value for this index changes from" 
-2.14 ^to 40.48. This change is due primarily to a rise in the average of 
the EXP sdiool from 7.50 to 8.68, but a decline in the average at the CON 
school from 8.65 to 8.40 also contributes. Thus, th^is pattern could be 
due to regression effect. Finally, the positive result obtained for the 
"Parent Supports Child's ;Leaming" index is again due to a pattern in. 
which there is an increase over tine at both the EXP and the CON school, 
but the increase at the EXP school is larger . It should be rioted that 
the coi^osition of this index makes it fairly similar to the index called 
"Adult Help for Student". Thus,, the fact that both indexes show a 
positive effect is in part due to this similarity. 

Overall i the pattern in the Oakland results seems fairly clear. There 
does seem to be an improvement in the various attitudes and behaviors of 
the parents in ttie EXP school in Oakland relative to that of the CON school. 
Of course, as with any of these findings , the observed result could be 
due to any number of causes other than an impact of the treatment. Hbwever, 
the consistency of results is strong enough that the general direction, 
if not each specific result, seems evident. 

The last of the individual site comparisons to be examined are those 
for San Antonio. Hiese results are presented in Table 111-89 . As in 
Oakland, thiey show a generally positive impact of the treatment. Also, 
as in Oakland, of the 12 indexes based on the Questionnaire, there are 
four positive shifts large enough to be noted, and only one negative shift 
of similar size. Three of the positive shifts are on indexes which also 
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showed a simiiar pattern in Oakland, These are: "Parent-Parent Contact", 
"Parent Attitude toward Child's Achievement" i anid "Parent Attitude toward 
Teacher"* The other index which shows a positive shift is "Parent 
Attitude toward School". The Questionnaire-based index which shows a 
negative shift is the "Adults Help Neighbor's Children" index* 

The observed shift in the Questionnaire-based index of "Parent-Parent 
Contact" occurs through a rise in the average level at the EXP school from 
Time.l to Time 2j and an essentially constant level at the CON school* 
The average at the EXP school on this index is 0*95 at time I, and rises 
to 1.48 at time 2, while the average at the CON school is 1.04 at the first 
time and 1.07 at the second time. The shift on the index of "Parent 
Attitude toward Child's Adiievement" also occurs primarily through a rise 
in the level of the EXP school. On this index, the average among the 
Ep school pareaits at Time 1 is 5.20^ and this rises, to 5.41 at Time 2. 
The level at Time 1 aiw>ng the CON school parents Is 5.14, and this declines 
slight ly to 5.09. On the index for "Parent Attitude toward S the 
parents in the EXP school are lower at Time 1, 6.29 as compared to 6.93 
for the CON sihobl. At time 2, however, the iEXP school averajge is 6.63, 
and the average at the CON school is lower, only 6.55. The fourth posi- 
tive result, for the index "Parent Attitude t^^ is another 
escainple of a reversal of direction, That is, the EXP school is lower at 
Time 1^ and is higher at time 2. Most of these patterns do not seem 
very likely to be the results of any possible regression effect. 

the indexes based pii the Parent Interview show two positive results, ^ 
and one result in the direction opposite to tfiat predicted. As was the 
case in Oakland, the index "Adult Help for Student" and: ''Parent Supports 
Child ' s Learning" both show positive effects . Again, it should be noted 
th *■ these two indexes aire constructed so that they share some conqponent 
itemt,, and so this similarity of outcome is not surprising. There is 
one negative result in San Antonio on the Interview-based indexes. The 
index "Parent-School Contact" shows a decline at the EXP school relative 
to the CON school, the pattern on the "Adult Help for Student" index 
is that the average in the EXP school rises from 8.70 to 10.00, and the 
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average In the CON school decides from 9* 17 to 8*54* On the related 
index for "Parent Supports Child's Learning", the pattern is that the 
CON school Increases over time, but the EXP sdhool increases more* The 
negative shift on the "Parent-School Contact" index occurs because there 
Is a rise in the amount of contact at both schools over time, but the 
rise is greater at the CON school. This pattern seems likely not to be 
due to regression effects. 

The general conclusion for San Antonio from this analysis is tlxat 
the pattern here in general is similar to that for Oakland. The reported 
attitudes and b^avior of the parents seem to become relatively more favor- 
able In the EXP school than in the CON school over the course of the 
project. 

Model-Based Comparisons— Parent Questionnaire 
and Interview Results 

As with the earlier analyses , the model-based conqpaxisons here are 
pres^ted primarily as supplementary Infoi^ for the benefit of 
readers who wish to see them. However, titiey should not be interpreted: 
as reflecting very directly the natture of the. dt^act . In view of the 
considerable diversity between^ t results of different sites, the model- 
based comparisons are not clearly interpretable. For the parent results, 
there is a fairly high degree of consistency between the patterns obtained 
in Oakland and those ob tallied in San Antonio . this makes the Parent- 
Teacher model con^arisons somei^at more justifi£d)le, but they still do 
not add nudi5 to die general finding that the two sites both show positive 
impacts. The comparisons for the TeacherrOhly mbde^ for the Parent- 

Teacher model sites, and for the PTr-E versus TO-E group, as well as the 
overall cpiiq[>arlson of all ElOP schools against all CON schools, are^ presented 
on the following pages with a brief discussion of each. Detailed discus- 
sion is not provided, because the real inte^ is on the individual 
sites. 
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When examined for the^ TeacherrOnly con5)arison, of the 12 indexes 
based oh the Parent Questionnaire, f pur show a negative intact, and three 
show a positive impact. These results are shown in Table lil-90. The 
four indexes which shift in the direction opposite to the hypothesis are: 
"Adult Help for Students", "Parent-TeacherContact"^ "Parent Attitude 
toward Child's Achievement", and "Parent Interest in Child's Achievement". 
The three indexes which show a positive shift are: "Parent Attitude 
toward Schppl", "Adults Help Neighbor's Children", and "Parent Attitude 
toward Incentives to Parents". There is only one Parent Interview index 
showing a shift of note from Time 1 to Time 2, and that is the index for 
"Parent-Schopl Contact". These results for the Teacher-Only sites are 
recogniz^le as a composite of the results for Cincinnati and those for 
Jacksonville. There seems little to add to what has already been said 
in the discussion of the sites separately. 

The coD^arlson f or the Parent-T^^^ sites (Oakland and Sari Antonio) 
indicates four shifts worthy of note and in the direction predicted, arid 
one shif t pf note in the direction opposite to what was predicted. The 
pattern of results is presented in Table III-91 and is similar to that 
for Oakland and San Antonio individually. TherCz are increases in the 
Questioririaire-based inde^^ "Parerit'-Parerit Contact*', "Parent Attitude 

towM^d diild's Achievement", "Parent Iriterest in Child's Achievement", 
and "Parent Attitude toward School". The, only bbserved outcome contrary 
to the direction predicted is that there is a shift downward in the index 
•^Parent Attitude toward Incentives to Teachers ". 

The interview-based indexes for the PT coEparisbn show three out- 
comes in the predicted directionrwhich are large enqu^ to note," and no 
reversals that are noteworthy, the three positive results are for: 
'•Adult Help for Student", "Parent Satisfaction with Child's Achievement", 
^and "Parent Supports Child 's Learning" . These results again are similar 
to those already noted when the two sites were discussed individuallyi 
In principle, the estimates presented in these ta^ should prb^de more 
accurate information as to what the average effect of the iriceritives 
model would be on parent attitudes than would those of either site 

(Text resumes on page 111^229) 
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individixally* However, ivi view of the great amount of site-to-site 
variation in all these variables, including parent attitudes and reported 
behavior, it would be presumptuous to use these estimates as if they were 
particularly accurate. The main point about them is that the pattern of 
effects in the two sites at which the Parent-Teacher model was used is 
generally, although not exactly, similar. 

The coiiq)arispn of the EXP schools in the two Parent- Teacher sites 
with the EXP schools in the two Teacher-Only sites, aimed at indicating 
something about the marginal impact of the PT model, is presented in 
Table 111-^92. This comparison is iised in lieii of a more direct one, 
btit it should be remembered that it can be distiorted by differences between 
sites other than those du^ to the treatment, this comparison indicates 
a number of favorable shifts on the Questiomiaire-based indexes and on 
the Interview-based indexes, there are 5 shifts in the predicted direc- 
tion of the Questionnaire-based indexes, and one shift in the direction 
opposite to that predicted. On the Interview-based indexes, there are 
again five shifts in the predicted direction § and no shifts in the oppo- 
site direction occur; 

The last of the modelrbMed comparisons is that for all of the schools, 
incfuding the four EXP schools on one side and the four CON schools on 
the other. Because of the dissimilar patterns of intact observed among 
the TO and the J?T sdi6pls, this overall comparison can be expected to 
mask as much as it reveals; and the outcomes observed from it do indeed 
mask some of the more interesting results. This overall comparison is 
presented in Table 111-^93. 

Parent Questionnaire and Interview Results* *- 
Discussion 

The TOst interesting result arising from this analysis of the Parent 
Questioxmaire and Parent Interview data is th^ general pattern of a posi- 
tive impact of the incentives model at the two sites where the PT model 
was implemented . These results, while hot perfectly cohsistent from one 
site to the other, do show enough consistency to justify the conclusion 
that the academically-related attitudes^^^^ of the parents at 

(Text resumes on page III-- 234) 
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these two sites Wak^i^^^^^ In the predicted direction. 

These results must, of course, be interpreted with some caution. The 
relatively lov rate of response to the Parent Questionnaire, and the 
lessening of response rate from Time 1 to Time 2, both make direct generali- 
zations from these results somewhat risky. However, it would be possible 
to err in the direction of excess caution as well as in the reverse direc- 
tion. Despite the necessary qualifications, and keeping in mind the 
various limitations that have already been mentioned about the difficulties 
encountered in in^lementing the model and Informing parents Aoiit the 
project, these results are definitely grounds to justify further, more 
precise^ investigation. 
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CHAPTER IV 

t 

SUMMARY OF FINDINGS, CONCLUSIONS, AND RECOMMQJDATIONS 

Findings 

Impact Results 

Student Achievement itesults , Uie analysis of student achievenent 
was based upon the data from the Metropolitan Achievement Tests, adminis- 
tered in the fall and spring to both experimental and control students. 
In Reading there w. t an educationally significant increase in learning 
rate only in one of the four sites, Oakland. There were negligible 
increases in two other sites, and an apparent decrease in learning rate 
at the EXP school in the fourth site, Cincinnati. Thus, the results for 
Reading seem to be dependent as much on factors that are characteristic 
of the individual sites as on any general impact of either the TO model 
or the FT model. 

For achievement in Mathematics, there were increases in learning 
rate in Oakland and San Antonio, which were the two sites at which die 
Parent Teacher model was implemented. However, these results were due 
in part to the characteristics of the CON school at each site, and so 
may reflect the operation of influences quite unrelated to the incentives 
model. Nevertheless, the mathematics achievement results are suggestive. j 

Student A ttitude and Results . The data qbtaincd from, the two instru- 
ments used to measure student attitudes show no interpretable pattern. 
The results from the Student Questionnaire in a nuniber of cases are 
directly contradictory to those obtatoed from the Student Inter>rLew. This 
lACk of clear pattern is especially evident at Cincinnati and Jacksonville. 
In Oakland, there are several indexes from the Student Questionnaire 
which suggests some slight positive intact of the incentives model, but 
the corresponding indexes from the Student interview are equally strong 
in showing trends opposite to the direction predicted. In San Antonio, 
the results again are not at all clear, although there is a slight 
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tendency for student self^reported attitudes to decline relatively over the 
course of the year* 

•Hiese findings are essentially noninterpretable. A number of possi- 
ble explanations can be made, including both substantive and methodological 
factors* However, it must be rioted that this set of data does not indi- 
cate any positive impact of the incentives model on student attitudes; 
therefore, to that degree at least, the results do not confirm the general 
hypothesis. 

Student Bdiavibr Results > As with the data from the Student Question- 
luiire and Student Interview, the data collected from the school records 
on attendance rates for classroom do riot show ^y discernible trend over 
time. If there is any such trend;, it probably is in a regative diicectibn, 
because all of the comparisons made indicate that the direction of change 
is toward a deterioration of the relative position of the EXP school. 

The riates of attendance indicated by thiis data are quite high, probably 
uriusually high for schools serving a low-income, inner-city population. 
Thus, it may be that the attendaiice rate cbuid not have been inproved by 
any new school project, because it had already reached a ceiling. 

The results pbtaijied from the Classroom Observation of student 
behavior show rio consistent patterris of difference between the EXP and 
CON schools. Although there are some relative changes in one or another 
kind of behavior observed from Time 1 to Time 2, these changes turn out 
to be the result of changes in tjie CON school as much as in the EXB 
school. Therefpre, it is concluded that they do riot reflect any con- 
sistent pattern. 

Thus, the data gathered on the attitudes and behavior of the students 
do not reveal any clear impact of the incentives model. The explanations 
that might be given for this lack of pattern in the data are several. 
Most obviously, the lack of pattern inight reflect the fact that the incen- 
tives model made no difference whatever to the students. Alternately, 
it might be argued that the instruments used, and the circumstances of 
their administration, led to a lack. of sensitivity, so that the true 
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differences were not picked up* This latter argument loses some force, 
in view of the fact that- there are real inconsistencies and reversals 
in addition to the absence of positive results in the data. A third 
explanation is that die initial formulation of the problem was erroneous. 
It may be that students in the elementary^ and particularly t^e early 
elementary grades , simply do not respond in a stable way to any new fea- 
ture of their environment. In a sense , it might be argued that children 
of this age, as a rule, do not have distinguishable and somewhat consistent 
attitudes or behavior patterns relative to schooling. 

Teacher Attitude Results . As with the data from the Student 
Questionnaire and Interviews , the results obtained from the Teacher 
Questionnaires and: Interviews are not immediately clear. However^ there 
is considerably more pattern to the results than was found: in the Student 
data. 

In Cincinnati y there was no very strong patterhy but there were 
several changes in the direction opposite to what was predicted. For 
Cincitmatiy. it can be concluded that there was a slight negative impact 
of the incentives model on the attitudes of the teachers involved. The 
results In Jacksonville include some negative changes also , but there 
are addition a ninnber of positive changes indicative of what might 
be described loosely as classroom -Varmth". On balance , it would appear 
that the iiq>act in Jacksonville y while limited to certain variables, 
may have been positive. In Oakland , the results for the Teacher Question- 
naire and Interview show few changes , and these few changes are in both 
directions. Thus, there is tK> real basis for concluding that the incen- 
tiyes model had any specified intact on teacher attitudes in Oakland. 
Finally y in San Antonio, there were several changes in teacher attitudes 
over time.. One reversal of direction in San Antonio is notable, 
particularly because San Antonio , from iaost of the available evidence, 
seems to have been the site at which the implementation of the project 
encountered the least opposition among the teachers. The negative change 
in San Antonio was on the index which measures a teacher's attitude 
toward modem teaching techniques, such as individualized instruction and 
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supervised peer tutoring. 

One point worth noting is that the teachers' professional associations 
were the most self-conscious and strongest in Cincinnati, where the iiiipact 
was somewhat negative, and in Oakland, where there was little trace of any 
impact. There may well be a political context and social process operating 
that inhibits the change of individual teacher attitudes. . 

Teacher Be havior Results . The data on teacher classroom behavior 
show no clear patterns. In fact, its most striking aspect is the chaotic 
quality of the differences observed over time. Amid this noise one can 
only guess at some slight trends. For example, in both Cincinnati and 
Jacksonville, the changes over time reflect an increase in more traditional 
and authoritarian behaviors (punishing, criticizing, lecturing) , and also 
increases in some behaviors more characteristic of modern approaches to 
classroom organization (answering student questions , praising, playing 
learning games, etc.). 

In the Oakland results, the pattern again is very faint. However, 
if there is a pattern-here, it is in the direction of a slight increase 
in the use of the more traditional behaviors. Finally ^ in San Antonio, 
the pattern again is mixed,, although with some traces of an increase in 
traditionalism. It should be noted that there may be specific defects 
in die data collected on Sah Antonio classrooms, and so the results from 
it must be used cautiously. Also, it should be noted that the patterns 
observed in Oakland and San Antonio, which were the two sites where the 
Parent Teacher model was used, were in several instances, directly contra- 
dictory. That is, in Oakland, a particular behavior would show an 
increase over time, but in San Antonio, the same behavior would show a 
decrease over time. 

On balance, then, the results of the Teacher Observation data do not 
point to any clear inq>acts. Indeed, they tend to vindicate the several 
teachers whose reaction to the notion .of an incentives project was, in 
effect, that the offer of Incentives alone, without any substantive pres- 
criptions, would not cause them to do anything differently because they 
had been working as hard and as effectively as they were able . 
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Parent Attitudei Results . In contrast to the various results just 
described, the Parent Questionnaire and interview data do show some 
fairly clear patterns. The caution must be made that these patterns could 
well be due to methodological artifacts, or to real features of the situation 
at a particular site other than the incentives model. However, the results 
do appear and should not be ignored. 

For Cincinnati, there are few changes ^ and these are generally in a 
negative direction. The data for Jacksonville, on the other hand, reflect 
a larger number of changes over time. These are mostly in a negative direc- 
tion, and lio particular pattern ercrges that might account for thetn. 

The results for Oakland and San Antonio, however, as reflected in 
these two instniments, show a fairly sizable nunier of changes, almost 
all of which are in a positive direction. There is one clear exception 
in both of these sites, however. In both Oakioid and San Aiitonipi there 
is no increase in the amount of Parent-School Contact observed in this 
data. Thus, the finding seems to be that the introduction of the incen- 
tives model led to an increase in level of parent interest and involvement 
in the child's education, and perhaps also to a more favorable attitude 
toward the sdiool. It did not, however, lead to the kinds of joint ac^ 
vities that had been hypothesized. This might be attributable to the delays 
encountered in delivering the contracts to the parents for their signature. 
Although there was a great deal of talk about offering incentives to parents, 
they may have taken a wait-and-see attitude before doing anything with the 
schools or other parents. 

Methodological Results 

Introduction . This summary of the methodological issues that emerged 
in the course of carrying out the implementation of the project, the data 
collection, and the analysis of the data, will be presented under the 
following headings: (1) measurement of academic achievement; (2) design 
features of the project; (3) attitude and behavior measuring instruments; 
(4) inplementation procedures; and (5) general considerations. 
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Measurement of Academic Achievement ♦ Over the past several years, 
because there has come to be an increased use of standardized achievement 
tests as Instruments for evaluating the success of various educational 
programs, Issues have emerged about their methodological properties and 
their limited applicability* In this project, many of these issues were 
recognized* Some were dealt with at the planning stage, others in the 
analysis* On balance, however, the issues remain sufficiently unresolved^ 
both in this project and in general, that some caution in interpreting 
these results must be employed. 

The problems include, for exaiq)le, the possibility that the use of 
aii inappropriate level of a test will cause the obtained score to be 
Influenced by a ceiling or a floor effect. To deal with this possibility, 
the design of the project implemented what has been called multilevel 
testihg. That is, each child in a classroom was aininistered the level 
of test that most closely matched the best estimate of his personal level 
of Reading achievement. Thus, the ceiling and floor effect should be less 
serious in this project than in most uses of mass testing. A second set 
of problems surrounds the choice of score format to be used in analyzing 
the results. In this project, it was decided to conduct the analysis using 
not the grade equivalent scores^ whidi have a number of undesirable 
features conmented on by recent writers (e.g^.^ Coleman and Karweit, 1972)_, 
but rather to use a score called the "standard score". This score, 
supplied by the test publisher,, is constructed so that it is in equiper- 
centlle hoanned scale score, comparable across all forms and levels and 
suggested by the publisher as suitable for measuring growth. 

Some additional issues emerged and were addressed in the actual analy- 
sis. One of these was the possibility of what has been called "fan spread". 
This means that students who begin with a high initial score tend to grow 
at a faster rate than students who begin at a lower score* Thus, over a 
period of time, the distance between the score of a very able child and 
the score of a less able child tends to increase. This phenomenon compli- 
cates analysis considerably and can distort the apparent conclusions. In 
this project, It was addressed and found to be of relatively small severity. 
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It is not likely to have created any sizable distortion in the conclusions. 
However, the whole topic is a new one<» and new techniques are being developed 
by a number of researchers. Thus, this issue may later be a worthy subject 
for further analysis. 

Another technical issue involved in the analysis of the achievement 
test data was the possibility of what is called regression to the mean. 
This phenomenon, in which the scores of an initially high scorer tend to 
drop back toward the average of the group from which he was drawn, 
operates not only for Individual students, but also for social units such 
as classrooms, grades, and schools. Because of the design of the project, 
there was ho way to determine-precisely the severity of regression to the 
mean. It may well have contributed to some degree to a distortion. This 
distortion would be such as to make the treatment school appear more 
successful than it really was. Thus, this distortion would, if any things 
contribute to aci over-positive interpretation of the results* This fact 
should be kept in mind when examining the achievement test results* 
However, the magnitude of the distortion due to the regression effect is 
unlikely to be large enough to change the general direction of the con- 
clusions or dieir educational significance. 

Two other issues of a methodological nature concern the problems of 
reducing the amount 'of noise (in the technical sense) present in the 
achievement data. In the analysis of this project, no direct attempt was 
made to reduce this noise, but the desirability of doing so (by means of 
estimates of reliability, corrections for attenuation, and perhaps the 
use of predictor variables in some well-defined model) is clear. The 
limited amount of relevant data and the time pressures involved for com- 
pleting the analysis precluded that kind of effort for this report. 

Design Features of this Project . There were several basic features 
of the design used for this project which impose limitations on the kinds 
of Inferences which can be drawn. Two of these are central. First, there 
was no use of the principle of randomized assignment of treatment to units. 
Second, the number of experimental units (i.e.^ schools) was quite small. 
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For these reasons, it must be made clear that the obsenred differences nay 
be due not to the impact of the incentives treatment but to idiosyncratic 
differences between individual schools and sites. A third feature of the 
design was that the four sites which were chosen are relatively hetero- 
geneous. This represents a strategic decision that is inherently difficult. 
That is, the use of a heterogeneous sample such as was done here allows 
one to make generalizations more widely if the pattern in the data is 
clear. However, if the pattern in the data turns out to be not uniform, 
then this design makes the inference less defensible as to what features 
of the population may have created those differences. 

Attitude and Behavior Measuring Instruments . The instrument used 
to measure academic achievement was chosen from among several well known 
and widely used standard achievement tests. In this sense, the choice 
was honcontfoversial, altJiough at a more specific level there are differences 
between these several tests which make the choice an iin)ortant one. How- 
ever, for the instruments used to measure the attitudes and behaviors of the 
various persons- involved in the school,, the situation was quite different. 
For these objectives, there were no readily available instruments which 
were fully appropriate . Thus new instruments , in many cases adopting 
formats from instruments used in other research, were developed. The 
development aimed at producing a siin)le, easily adndnis tared instrument 
which would have immediate and direct relevance to the Bsearch questions. 

A second feature of the instrumentation was an attempt to use the tri- 
angulation notion to pin down the actual changes in attitudes and behavior. 
That is, in several areas, a single broad concept was to be measured by 
several instruments of different kinds. This approach, desirable as it 
was, was handicapped by the short time available for development and 
pretesting of the instruments , and further handicapped by the considerable 
delay in obtaining OE and 0MB forms clearance for the questionnaires. 
The analysis of the data from these varied instruments indicated a number 
of unexpected inconsistencies. These inconsistencies create limitations 
on the confidence that can be placed on any particular interpretation. 
However, they also indicate that this is an area which deserves further 
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exploration and attention in future research. 

Implementation - Procedures . In general, the implementation of the 
original design can best be described as delayed and partial. The 
details of these delays will be mentioned in the succeeding section, where 
some of their iiq>lications are used to draw recommendations for future 
practice. However, at this point it can be noted that the problems in 
implementation led to two major consequences. First, the treatment siin)ly 
was not tried at its full strength. That is, the contracts with the 
participating districts were not signed until after approximately Italf of 
the school year had already passed. For some of the parents, the first 
real awareness of the project came at the tiiOB they were asked to fill out 
the firfift Parent Questionnaire. This was in March of the school year, 
after about two-thirds of the year had passed. Thus, whatever inyacts 
are iseen here should be regarded as the ^b^ of a seriously attenuated 

treatment. Second, there were delays encountered at two sites because the 
building principals were unwilling to have the evaluation begin until they 
thought their schools were prepared. 

General Considerations . The general point to be made about the 
methodology here is simply that the results may well be due to a variety 
of factors other than the pure impact of a clean incentives model on 
schools which are otherwise known to be equal. The various methodological 
problems introduce noise, which makes any patterns difficult to detect, 
and perhaps creates actual bias. Thus, the data analyzed here should be 
regarded much as one would regard testimony in a courtroom. The witness 
is not deliberately trying to create one or another impression, but to 
tell the facts as he perceived them. Yet, it must be the jury's 
respunsibility in weighing the evidence on data to take into consideration 
the limitations oh the witness's ability to perceive what happened and 
on the accuracy of his recall. These results, then, like most in social 
science, must be interpreted as part of a larger context. 
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Conclusions 

The Impact of Incentives 

Introduction > Across all participants and variables, a consistent 
positive trend is not immediately apparent. The influences at work in 
the four sites and eight buildings — traditions, role relationships, the 
individual principals — ^were complex and complicating factors in the 
analysis of results. However, the analysis did indicate that these and 
other sources of variance were at least as powerful as the introduction 
of the incentives models. Therefore, the conclusions offered about the 
impact of the incentives models are highly tentative. 

single-Site Based Conclusions . 

o In Cincinnati, the offer of incehtives to teachers, the Teacher 

Only model, had a negative impact; Student achievement in Reading 
and Mathematics declined relative to the control school. Teacher 
attitudes appear to have become more negative. Parent 'attitudes 
became very slightly more negative. 

o In Jacksonville, the offer of incentives to teathers, the Teacher 
Only model ,had a mixed effect both positive and negative . 

o In Oakland, the offer bf incentives to parents and teachers, 
the Parent Teacher model, had a positive impact. Student 
achievement in Reading and Mathematics increased relative to 
the control school. Parental attitudes became more positive. 
However, teacher classroom behavior became slightly more 
traditional. 

o In San Antonio, the offer of incentives to parents and teachers, 
the Parent Teacher model, had a slight positive effect. Student 
achievement in Mathematics increased relative to the control 
school. However, the students* attitudes were slightly more 
negative. Teacher attitudes were somewhat more positive, but 
they were also more traditional. Teacher classroom behavior 
became more traditional. Parental attitudes became more favorable. 
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Model'Based Conclusions > Although it has been a repetitive theme 
throughout this analysis, it is inportaht to reaffirm the conviction chat 
the conclusions that follow are based on a very small sample of sites, 
buildings, and participants. Furthermore, the Teacher Only and Parent 
Teacher models were not fully or rigorously tested in this demonstration 
project. 

o Incentives to teachers, the Teacher Only model, had an overall 
negative impact. At one site the impact was markedly negative. 
At the other site, the iiq>act was both positive and negative. 

o Incentives to parents and teachers, the Parent Teacher model, 
had an overall sligihtly positive intact* At one site, the 
results were mostly positive with indications of some slightly 
negative results. At the other site, there was a slightly 
positive Ijnpact. 

The Logistics and Manafeement of Incentives 
Arrangements 

introduction . In addition to the conclusions reached on the basis 
of the quantitative analysis of the data, there are some further conclu- 
sions to be drawn on the basis of the esqperiences of this project. 
Generally, these conclusions deal with the kinds of difficulties which 
arise In getting an Incentives program established and are consistent 
with the project's objective of demonstrating the feasibility of incen- 
tive arrangements In education. 

Payment Formulas . The first of the logistical and management prob- 
lems is one that arises when tlie payment formula is established. As labor 
negotiators are well aware, it is often a long step from an agreement in 
principle about how a payment rate will be determined to an actual set 
of tables and rules for use In calculating the payment. When incentives 
are used in education, the problems of establishing a payment formula 
are Increased because of the lack of consensus among the parties involved 
as to what should be the output upon which they should be measured; 
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the lack of agreement as to what influences in addition to their own 
efforts affect the lavel of output; and finally by disagreements about 
the technical problems associated with the measurement of academic achieve- 
ment. In this project, the initial approach was to calculate a single 
expected growth benchmark for each grade at each school and then to offer 
payment to any teacher according tothe^amount.by which that teacher's class- 
room exceeded the benchmark. As a result of objections raised by some 
teachers, the decision was made to revise this procedure and to develop 
growth benchmarks for each classroom. Thus, each teacher was working 
against a more appropriate criterion. The management implication of these 
events is that even apparently simple and direct incentives models such as 
these are quite complex when introduced into school settings. 

School District Data M anagement . The second problem that arose in 
this project has been encountered by other researchers. In general, school 
district records are not comprehensive or maintained at a useful level 
of aggregation. Thu^ research questions that focus upon the building, 
grade, or classroom level of aggregation cannot be asked of school district 
data. Furthermore, those data which are maintained; in the buildings are 
either inaccurate or inconsistently collected and maintained. These 
limitations of available school documents, both for the previous and 
current years, reduced the validation that should have been done when 
unanticipated results were found in this project. The limitations also 
made the early stage of project implementation, the selection of sites and 
the choice of school buildings within sites, less objective than it should 
have been. The conclusion is that unless and until the management of the 
school districts' data of record is vastly improved the expectations of, 
and generalizations from, field projects of this type should be extremely 
modest. 

School Di strict Authority . Although it was not a major difficulty in 
this project, it should be noted that establishing a full working liaison 
with the district and defining areas of authority can be very difficult. 
This is not intended to be a criticism. Rather, it is meant as a reflec- 
tion upon the functions and structure of school districts and the 
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researcher's understanding of them. Like many public institutions, school 
districts, for good reason, are organized to avoid mistakes. Thus, 
management functions and authority are shared by many people within the 
administrative hierarchy. No one person has an unchallengeable discre- 
tionary contirol over the kinds of resources that must be quickly mobilized 
if a project of this sort is to be iB5)lemented. For example, the Cincinnati 
board overruled its management on the type of incentives model to be used. 
And, in Oakland, the original control school faculty withdrew from the 
project at the last minute. In effect, management was overruled twice in 
this project, once by the ^'board of directors" and once by the "workers". 

Furthermore, because autfiority is diffuse and subject to numerous 
checks «id balances within school districts, the fact that an agreement 
or contract has been signed is no guarantee that the evaluation plan will 
be accepted. Thus, in Oakland and Cincinnati, building principals refused 
to allow data gathering to 5; E initiated until they felt that their schools 
were prepared for it. . One of the anomalies of power relationships in 
school districts is that, despite appearances, the building principals 
and their faculties have the potential for a great deal of autonony. It 
only appears that they are locked into a position of powerlessness because 
the school buildings have such limited opportunities -to make choices. 
When the opportunity arises, it is surprising how quickly this latent 
power emerges. 

The conclusion to be drawn is that two months is not enough time, and 
the approval of only the central office is not sufficient authority, to 
install a field project in a school district. Furthermore, the willing 
consent of the building principal and the faculty is the only assurance 
that the evaluation, so often a source of irritation to people who are 
atten?)ting to get on with the job, will be implemented as planned once 
the project has been installed. 

Parents and Schools . This project, at least in the two districts 
that had the Parent Teacher incentives model, necessitated conmunications 
with the parents that were different in kind and degree from the communica- 
tion procedures normally practiced by school districts. At a minimum. 
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it was required that all parents be infoxned of the project's 6b j.:ctive8 
and procedures. Ideally , a sustained and continual dialogue between the 
parents and the schools would have emerged as a desirable outcome. 

That the aininum requirement was barely satisfied and that the ideal 
conmmications pattern was not realized should not be a surprise. Schools 
are required by law and organized to teach , socialize and care for children. 
Parents 9 despite some populist suspicion of schools , are generally satis- 
fied that these outcomes are being achieved. Thus, the existing communications 
system between homes and schools consists of a ritualistically attended 
"open house" «tid report cards. This is a perfectly functional relation- 
ship, supported by a lengthy, tradition. Consequently, the iiq^iemcntation 
of a project which requires the mobilization of parents will strain the 
existing system beyond its potential if additional resources and time 
are not devoted to this problem area. 

OE Project Admtoistratioh . TWo administrative problems within OE' 
seriously inpeded the Implementation and evaluation of this project. First, 
the grants to the districts were delayed as they passed through the 
required internal approval process. Second, the questionnaires were 
inordinately delayed in the OE-OMB clearance process. The time constraints 
of this project were not very different from those of similar past and, 
presumably , future field efforts; therefore, these delays deserve some 
attention. It is an axiom of organizationiBil theory that authority must 
be commensurate with responsibility. In this project, this relationship 
did' not exist. The OE Project Officer found that he had the power only to 
persuade or cajole others to free die grants to the districts because of 
Insufficient authority. In the process of securing the grants, his project 
was delayed and the morale of the group that released the funds was not 
iiq>roved because the concept of incentives was not among their approved 
fiscal year objectives. The necessity for the review and approval of the 
questionnaires is acknowledged. However, the four to five month clearance 
process encountered in this project is questionable when the instruments 
or procedures are not materially altered. 
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Because of adoilnistrative delays » the impact of the incentives 
models was probably attenuated and the evaluation design vas certainly 
weakened. One can conclude that field experimentation of this type re- 
quires a significant reorientation of the priorities and consolidation 
of authority within the agency responsible for the evaluation. 
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Re commenda tions 



Discussion 

If the idea of incentives has any merit in public education, then the 
limitations of this project should be discussed before the recommendations 
are presented. First, the incentive models were intentionally narrow 
in their scope. Incentive plans can differ along four dimensions: the 
recipients, the timing, the criteria, and the type of incentive. This 
project offered incentives only to teachers or parents and teachers;, 
important other participants in the learning process such as principals 
and students were not included. Along the second dimension, timing, the 
incentive awards were not to be given until the end of the year, rather 
than offering interim progress payments. The criteria for payment were 
classroom reading and mathonatics standardized achievement test scores; 
other units of aggregation such as the grade level could have been used, 
along with many other outcomes and measurement instruments. Finally, the 
incentives were cash payments which are extrinsic rewards (and essentially 
demeaning according to the statements of some teacher groups). 

The second limitation is that the incentive models received less than 
a full exercise and evaluation in this project. In retrospect, the imple- 
mentation fell short in three areas. The school districts were not recruited 
into the project until the middle of the summer; hence the buildings were 
not selected with the consent of the teachers and parents who would parti- 
cipate. Second, the grants to the districts were delayed; therefore, the 
parents and teachers did not have contracts to sign until half way through ' 
the school year. Finally, administrative delays prevented several of 
the evaluation instruments from being administered at the start of the 
project. Among the remaining instruments that were not administratively 
delayed, with the exception of the achievement tests, several were delayed 
by the building principals in two sites. 

Finally, when considering these recommendations, it should be remem- 
bered that the idea of incentives arouses latent but very powerful feelings 
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that are central to the American ethos.. The idea is more than just 
controversial on technical grounds: it is subject to political exploita-" 
tion. For this reason, the eaq)eriiaentation with, and advocacy of, incen- 
tives in education must be extremely cautious. Regardless of the results 
of this and other Incentives projects, it should be remeobered that incen- 
tives are not Ukely to be a panacea in education, aisy more than they have 
been In industry. Yet, they may well be, as in industry, a useful techni- 
que in certain forms and circumstances. The central point about incentives 
is that they are capable of great harm as well as benefit; therefore, they 
should be used with care. 

Recommendations 

Incentives in Education , 

1. Based iqjon the evaluation of this project and the evaluation of 
the OEO teacher Incentives projects in Stockton, California, 
and Mesa, Arizona, the Teacher Only model should not be further 
researched or advocated. 

2. Based upon the results of the evaluation of the Parent Teacher 
model, it is recomncnded that field research of this model 

be l]q>lemented. 

3. Based upon this project and some knowledge of the OEO perform- 
ance contracting experiment, it is recommended that any future 
field research into Incentives in education recognize that incen- 
tives are an inextricable part of the accountability movement 

in public education. Consequently, Incentives iin)licitly involve 
politics and changes In the distribution of control within school 
buildings and the school district. The minimal elements of such 
a recognition in either field experimentation or advocacy of 
incentives would be: 

o Parents, teachers and others who are the participants must 
have the opportunity to choose freely to participate. 
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o The participants must have the opportunity to participate 
actively in the design of the incentives projects, includ- 
ing such areas as the types of rewards, their timing, etc. 

o The participants must have* a degree of control over their 
working ;environment that is commensurate with the degree 
to which they are being held accountable. 

o The participants must be provided the resources that they 
feel are necessary for them to meet their commitments. 

o The school building is the accountable unit and all pro- 
fessionals in that unit — teachers, the principal, super- 
visors — share the responsibility. 

o The criteria by which performance is measured and rewarded 
should reflect the range of student outcomes for which the 
school personnel are responsible. 

The Conduct of Field Projects . 

4* At a minimum^ field projects should be operated on a three-phase 
basis, with at least one year for each phase. Phase 1 would be 
devoted to recruitment, iiq)lementation, instrtment development, 
design refineioent, and the collection of baseline data. Phase 
2, the second academic year, would involve the actual operation 
of the project and the collection of the primary evaluation 
data. Phase 3 would include the analysis of the primary data 
and the conduct of follow-up studies. 

5. The agency that designs and initiates a field project and the 
agency that iiq>lements the project, generally, the school 
district, should both participate in funding the project. Past 
and present field projects have been funded exclusively by 
the Federal Government, with the districts providing funds or 
services that are only opportunity costs. The districts do not 
have a direct fiscal incentive to see that the projects are 
operated smoothly or evaltiated fully. One way to gain a responsible 
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commitment from the cooperating districts would be to require that 
the school districts fund a portion of the costs of the project. 

If the Office of Education is going to continue to operate field 
endeavors that are based upon a nationwide experimental design 
and that require a complex evaluation plan, the authority of 
the officials responsible for the project must be extended to all 
areas of the project *s operation. Furthermore, the responsiveness 
of middle level OE officials to such re^^earch efforts must be 
addressed and their cooperation guaranteed. 
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INCENTIVES IN EDUCATION 
STUDENT QUESTIONNAIRE 



(NOT IN ORIGINAL FORMAT) 





2. When I think about learning to do arithmetic problems, I feel like: 



INCENTIVES IN EDUCATION 
STUDENT QUESTIONNAIRE 




3. Vnien I think about reading, I feel like: 





4. When I think about doing arithmetic problems, I feel like: 



INCENTIVES IN EDUCATION 
STUDENT QUESTIONNAIRE 





Vhen I think of my arithmetic teacher, I feel like: 



INCENTIVES IN EDUCATION 
STUDENT QUESTIONNAIRE 




7. Nhen I think about learning other subjects, I feel like: 




8. When I talk to teacher in school the teacher looks like: 



INCENTIVES IN EDUCATION 
STUDENT QUESTIONNAIRE 




10. When one of the kids asks a question, the teacher looks like: 



INCENTIVES IN EDUCATION 
STUDENT QUESTIONNAIRE 




11. When another kid in my class makes a mistake, I feel like: 




12. When we try to have fun in class, our teacher looks like: 



INCENTIVES IN EDUCATION 
STUDENT QUESTIONNAIRE 




J. The balloon-child goes to a good school; the flag-child goes to a 
bad school. 




/ 



\ 



o 



J 




14. 



The balloon-child is not good at doing arithmetic nroblems; the flag- 
child is good at doing arithmetic problems. 
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INCENTIVES IN EDUCATION 
STUDENT QUESTIONNAUO; 




16. The balloon-child likes to go to school; the flag-child does not like 
to go to school. 



INCENTIVES IN EDUCATION 
STUDENT QUESTIONNAIRE 




17 • The balloon-child likes to work at school with arittunetic problems; 

the flag-child does not like to work at school with arithmetic problems. 




18. The £lag--child likes to read at school; the balloon-child does not 
like to read at school. 
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INCENTIVES IN EDUCATION 
STUDENT QUESTIONNAIRE 





INCENTIVES IN EDUCATION 
STUDENT QUESTIONNAIRE 




23. The balloon-child writes notes to other people; the flag-child does 
not write notes to other people. 




24. The flag-child helps other kids outside of school with their school- 
work; the balloon-child does not help other kids outside of school 
with theiiT schoolwork. 
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INCENTIVES IN EDUCATION 
STUDENT QUESTIONNAIRE 




J 



V 



O 




25. The balloon-child helps other kids at school with their schoolwork; 

the flag-child does not help other kids at school with their schoolwork. 



I 




J 



V 



O 




J 



26* The balloon-child wants all the kids to do well in school; the flag- 
child does not care how the kids do. 



ERIC 
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INCENTIVES IN EDUCATION 
STUDENT QUESTIONNAIRE 




J 



o 




27* The flag-child wants to do well in school; the balloon-child does not 
care how well he does in school. 




o 



/ 



V 



o 




28. The balloon-child wants to be good at doing arithmetic problems; 
the flag-child does not care. 
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INCENTIVES IN EDUCATION 
STUDENT QUESTIONNAIRE 




\ 



o 



J 




29. 



The balloon-child wants to be a good reader; the flag-child does 



not care 



ERIC 




O 



J 



o 




30. 



Most people think the flag-child is good. Most people think the 
balloon-child is bad. 
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INCENTIVES IN EDUCATION 
STUDENT QUESTIONNAIRE 




y 



V 



O 



J 




31. The balloon-child expects to be a good reader; the flag-child does 
not expect to be a good reader. 




/ 



V 



O 



J 




ERIC 



32. It is hard for the balloon-child to learn things; it is easy for the 
flag-child to learn things* 

A^^IT ~ 



INCENTIVES IN EDUCATION 
STUDENT QUESTIONNAIRE 




/ 



V 



o 



J 




33. 



The balloon-child expects to go to high school; the flag-child does 
not expect to go to high school. 




/ 



V 



O 



J 




34. 



The flag-child can do the things he wants to; the balloon- 
child cannot do the things he wants to. 



ERIC 



A-18 



INCENTIVES IN EDUCATION 
STUDENT QUESTIONNAIRE 




The balloon-child expects to be good in arithmetic; the flag-child 
does not expect to be good in arithmetic. 




The balloon-child gives up easily on his work; the flag-child 
finishes his work. 
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INCENTIVES IN EDUCATION 
STUDENT QUESTIONNAIRE 




37. The balloon-child does not like himself most of the titne; the flag* 
child likes himself most of the time. 




38. The balloon-child wants to go to high school; the flag-child does 
not want to go to high school. 
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INCENTIVES IN EDUCATION 
STUDENT QUESTIONNAIRE 




y 



V 



O 




The flag-child wants to get training for a gocri job; the balloon- 
child does not care. 




The balloon-child will get what he wants because of luck; the 
child finishes his work. 
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INCENTIVES IN EDUCATION 
STUDENT QUESTIONNAIRE 




y 



V 



O 




41. The balloon-child expects to be trained for a good job; the flag- 
child does not expect to be trained for a good job. 




y 



O 



y 



er|c 



42. The balloon-child will get the job he wants. The flag-child will 
not get the job he wants. 
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OMB 51-571055 
Approval expires 7/31/72 

WmE_ DISTRICT SCHOOL 



TEACHER QUESTIONNAIRE 

This questionnaire contains two parts numbered Part I 
and Part II. The questions in Part I are to gather 
background information. Part II questions will be asked 
only on this first administration of the questionnaire. 
During the second administration later this year^ you 
will not be required to complete Part II again. 

DIRECTIONS: For each question, please place a check mark in front of 
the number of the response that is closest to your answer 
of place a number in the blank space provided. Answer 
all questions. Thank you. 

PART I OF THE QUESTIONNAIRE ASKS ABOUT lOUR SCHOOL- 
RELATED ACTIVITIES. 



1. How much time per day (average) do you spend outside the normal 
school day on professional preparation (lesson planning, etc.)? 

^ hours 

2. What percentage of your time do you feel should go to: 



top fourth of your students In arithmetic % 

second fourth of your students In arithmetic _ % 

third fourth of your students In arithmetic X 

bottom fourth of your students In arithmetic " X 

3« What percentage of your time do you feel should go to: 

top fourth of your students tn reading X 

second fourth of your students In reading X 

third fourth of your students In reading x 

bottom fourth of your students In reading X 



QUESTIONS 4 TO 28 ASK YOU ABOUT YOUR ATTITUDES TOWARD STUDENTS, TEA^ 
CHING, AND THE SCHOOL. Please place a check mark in one of the boxes 
along the line to indicate how you feel about each question. There 
are no pr eferred answers to these questions. In the example below, 
the check mark ehows that the person answering does not feel very 
strongly about television programs one way of the other. 

EXAHPLE: I think television programs today are: 

awfu3 | t I t t t great 

4* My school: 

could stand a lot could serve as a 

of Improvement i I i t i i model to others 
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5. Considering only their motivation to learn, vsy students are: 

very poorly motivated lit j i i very highly motivated 

6. Considering only their academic a bility , my students are: 

very limited i i i i i , very capable 

7. Considering only their behavior, my pupils behave: 

very poorly I I i i i i very well 

8. I consider my work as a teacher: 

not gratifyin g t I I i t gratifying 

9. The offer of incentives to a teacher based on the achievement of his or 
her students is: 

unlikely to lively to increase 

Increase achievement ! t I | i jachievement 

10. If I want to use new teaching methods or materials, my principal is 
likely to: 

oppose m ^ I I I I j assist me 

11. The role most of my pupils' parents play in their children's education is 

of no significance, i » i » j of great significance 

12. My feeling about transferring to .another school is: 

very opposed i i t t , j very favorable 

13. My feeling about my status in this community (school attendance area) 



is: 



dissatisfaction j I I i | | sat<sfarr4nn 



14. Letting children move around in the classroom and talk to each other: 

prevents learnin g I I I i i fosters learning 

15. The offer of incentives to parents based on achievement of their child's 
class is: 

not propeq lilt ip rooer 

16. Please indicate your attitude toward your school: 

strongly negativ e i t | | i stronelv positive 

17. Faculty meetings at my school are: 

counterproduc t iv q i i i | ip roductlvo 

18. Efforts of the faculty at my school to assist one another are: 

counterppoduotiv q I I i i ip roductive 

19. My principal's attitude towards new teaching methods and materials is: 

negativ et I i r i ip ositive 

20. Please indicate your attitude toward your pupils' parents: 

strongly negativ q | t | | t stronelv positive 

21. The offer of incentives to a teacher based on the achievement of his 
or her students is: 

not proper ! I I i | ip mppr 
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22. If a child is allowed to proceed at his own rate during a year's time, 
he is likely to: 

learn less | | ^ i ^ ^l earn more 

23 • My feelings about having faster pupils tutor slower pupils in my class- 
room are: 

opposed I I I t I jf avorable 

24. The offer of incentives to parents based on the achievement of their 
child's class is: 

unlikely to increase likely to increase 

achievement j | | i i ^a chievement 

25.. Strict discipline in the classroom is an important part of a child's 
education: 

I strongly disagree t | t it tl strongly agree 

26. Please indicate your attitude toward your pupils: 

strongly negative | | r j i ts trongly positive 

27. Having faster pupils tutor slower pupils is likely to be: 

very ineffective \ \ \ t i iv ery effective 

28. Individualized and self-paced Instructions are likely to be: 

very ineffective j I | i t ^v ery effective 
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PART II 



DIRECTIONS: For each question, please place a check mark In front of the 

number of the response that is closest to your answer. Thank you. 



1. What is your sex? 

1. Female 

2. Male 



2. How old were you on your last birthday? 
^ 1. under 26 

2. 26-35 

3. 36-45 

4. 46-55 

5. 56 or older 



3. What will be your base-pay salary from this school system this yearl 

1. $4000-4999 

2. $5000-5999 

3. $6000-6999 

4. $7000-7999 

5. $8000-8999 

6. $9000-9999 

7. $10,000-10,999 

8. $11,000-11,999 

9. $12,000-12,999 

10. $13,000 or more 



4. What is your emplojrment status in this school system? 
1. Tenured appointment 

2. Non-tenured appointment 

5. What is the highest academic degree that you hold? 
1. None 

2. A degree based on less than 4 years of work 

3. A Bachelor's degree 

4. A Master's degree 

5. Professional or Specialist diploma 

6. A Doctorate 



6. How many credits of graduate work have you had beyond your highest 

academic degree? 

1. None 

2. 1-10 credits 

3. 11-20 credits 

4. 21-30 credits 

5. 31+ credits 



7. As of June, 1971, what was the total number of years that you have been 
a full time teacher? 

years 

8. How closely do you live to the school building in which you teach? 
1. Within the school building's normal attendance zone. 

2. Not in the building's attendance zone, but within one mile of 

the school. 

3. Inside the school district boundary. 

4. Outside of the school district. 
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.How did you happen to be assigned to this particular school rather 
than some other school in this district? 

!• I asked to work in this- school. 

■ , 2. I was placed in this school. 

^O*^?!?" "^^^"^^^ ^ transfer at any time from this school in school year 

This is my first year teaching in this building. 

2. No * 



3. Yes 



In the last year (70-71) approximately how many school days were you 
absent from this school? (Leave blank if you did not teach in this school 



last year) . 
days 
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CHILD'S GRADE 



INSTRUCTIONS: 



DIRECTIONS: 



0MB N° 51-571056 
Approval expires 7/31/72 



SCHOOL 



DISTRICT 
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PARENT QUESTIONNAIRE 
PART I 

at home who %s most responsible for the daily cdre and 
h7mT""7hZld^ 'l''^ questZnnate 
home a questtonnaire please complete only one set wLn 

Thl^hhrX'TrotZl^'i^^ '\l ^'^^^^ "yolrohilt' it7e-ans 
InLi.li Jr°'*9>it home the questionnaire you are now 

and^il iart ri ^^ff ^""-j- two parts /numbered I 

ana ii. Part II wtll be asked only on the first Question- 
natre. Your cooperation is appreciated. Thank you 

QUESTIONS 1-18 ASK ABOUT YOUR ATTITUDES TOWARD SCHOOL, 

Pjease place a check mark in one of the boxes along 
Irne to tndtcate how you feel about each question. 
r^gre are no preferred n. swers to ±h... questions 
In the example below, th e check mark shows that l^^ ^ 
person answering does not feel ve^y stZZlu about 
telev^s^on programs one way or the\thev. 

Example 1: I think television programs today are: 
bad no opinion good 



the 



en 



czi 



In the example below, the check mark shows that the person answering 
does not like what is shown on television programs • 



Example 2i 



I think television programs today are: 
bad no opinion good 



In the below example, the check mark shows that the person answering 
does like what is shown on television programs. 



Example 3: I think television programs today are: 



bad 



tzzi 



My child *s school: 
could stand 
Improvement 



no opinion 
no opinion 



good 



could serve as a 
model to others 



My child's teacher is: 
one of the worst 
in the school 



ho opinion 



one of the best 
in the school 



How do you feel about your child *s school? 

dissatisfied no opinion satisfied 



LZl 
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4. Offering teachers more money if their pupils learn more is: 
not likely to 

work no opinion likely to work 



5. A good way to get my child to do well in school is to punish 
him when he does poorly* 

I disagree no opinion I strongly agree 

i=! □ cm 

6. When it comes to my child's education his teachers: 

do not care no opinion do care a lot 

n [ 



7. My feeling about discipline at my child's school is: 
. dissatisfied no opinion satisfied 



CD 



8. Encouraging parents with money if their child's class learns more 
is: 

not proper no opinion proper 



9. To me, my child's ability to do arithmetic well: 

does not matter no opinion is very important 



10. To me, my child's ability to read well: 
does not matter no opinion is very important 

□ [ZD □ 

11. How do you feel about your child's teacher? 

dissatisfied no opinion satisfied 

CD CD CD 

12. Offering teachers more money if their pupils learn more is: 

not proper no opinion proper 



CD O 



13. How is your child's teacher doing in teaching arithmetic to him? 
poor job no opinion good job 



14. How is your child's teacher doing in teaching reading to him? 
poor job no opinion good job 

CD I 



15. A good way to get my child to do well in school is to praise 
him when he does well* 

I disagree no opinion I agree 



n [ 



16. Offering parents money if their child's class learns more is: 
not likely to 

work no opinion likely to work 



□ 
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I 17. How well is your child doing in arithmetic this year? 

\ poorly no opinion well 

□ □ CZI 

» 18. How well is your child doing in reading this year? 

poorly no opinion well 

(=□ czi (=□ 

QUESTIONS 19 TO S3 ASK ABOUT YOUR INVOLVEMENT WITH 
YOUR CHILD'S EDUCATION. 

? 

I DIRECTIONS: Please circle the number of the response that is 
I closest to your answer. 

19. How often does your child get help with arithmetic from 

someone who lives in your home (an adult or another child)? 

1. About every day 

2. About once a week 

I 3. About once a month 

I 4. A few times a year 

I -5. Never 

I 6. Don't know 

r 

I 20. How often does your child get help with arithmetic from an 
I adult (other than the teacher) who does not live in your home? 

I 1. About every day 

I 2. About once a week 

3. About once a month 
I .4.. A few times a year 
I 5 . Never 

I 6. Don't know 

I 21. Hov often does an adult in your home help other people's 
I children with arithmetic? 

I 

i 
f 

I 
i 



22. 



23. 



1. 


About 


every day 


2. 


About 


once a week 


3. 


About 


once a month 


4. 


A few 


times a year 


5. 


Never 




6. 


Don't 


know 


How 


often 


does your child 


child? 




1. 


About 


every day 


2. 


About 


once a week 


3. 


About 


once a month 


4. 


A few 


times a year 


5. 


Never 




6. 


Don't 


know 


How 


often 


does your child 


child? 




1. 


About 


every day 


2. 


About 


once a week 


3. 


About 


once a month 


4. 


A few 


times a year 


5. 


Never 




6. 


Don't 


know 
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How often do you encourage your child to do homework? 
!• About every day 

2. About once a week 

3. About once a month 

4. A few times a year 

5. Never 

6. Don't know 

How often does your child get help with reading from someone 
who lives in your home (an adult or another child)? 

1. About every day 

2. About once a week 

3. About once a month 

4. A few times a year 

5 . Never 

6. Don't know 

How often does your child get help with reading from an adult 
(other than the teacher) who does not live in your home? 
!• About every day 

2. About once a week 

3. About once a month 

4. A few times a year 

5. Never 

6. Don't know 

How often does an adult in your home help other people's childr 

with reading? 

1* About every day 

2. About once a week 

3. About once a month 

4. A few times a year 

5. ' Never 

. 6. Don't know 

How often does your child get. help with reading from another 
child? 

1. About every day 

2. About once a week 

3. About once a month 

4. A few times a year 

5 . Never 

6. Don't know 

How often does your child give help with reading to another 
child? 

1* About every day 

2. About once a week 

3. About once a month 

4. A few times a year 

5 . Never 

6. Don't know 
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I 30. When your child wants to do his schoolwork at home, can he have 

1 use of the following things (please place a check mark next to 

I those things that he can use). 

I !• A quiet room 

i 2. A desk or table 

j 3. A dictionary 

4. An encyclopedia set 

5. Blank paper 

6. Pencils, pens 

31. About how many hours a week, if any, does your child watch tele- 
vision (please write in the number). 

hours 

32. About how many hours a week, if any, does your child watch the 
following children's television program (please write in the 
hours). 

Sesame Street hours 

33. About how many hours a week, if any, does your child watch the 
following children's television program (please write in the 
hours). 

The Electric Company hours 

QUESTIONS 34 TO 38 ASK ABOUT YOUR COMMUNICATIONS WITH 
THE SCHOOL. 

DIRECTIONS : Please place a number which approximates your activity 
for each question. 

34. In the past two months, I have had about meetings with my 

child's teacher. If no activity has taken place then mark a zero. 

35. In the past two months, I have had about telephone conversa- 
tions with my child's teacher. 

36. In the past two months, I have had about written notes from 

my child's teacher. 

37. In the past two months, I have been to meetings of parents 

at the school. 

38. In the past two months, I have been to meetings with parents 

concerning school but not at the school. 
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PART 11 

QUESTIONS 1^7 ASK ABOUT iOU AND YOUR FAMILY. 

DIRECTIONS: For each queetion^ please circle the number of the 
reaponee that is closest to your answer. 

1.. Excluding the child whose nane appears on the first pa^e 
how many children do you have at home? 

1. 1 to 2 

2. 3 to 4 

3. 5 to 6 

4. 7 to 8 

5. 9 or more 

2. What is your relationship to your child whose name appears 
on the first page? 

1. Father 

2. Mother 

3. Foster parent 

4. Family relative 

5. Other 

3. The approximate total family income for 1971 was (include all 
sources of income - work, gifts, welfare, social security and 
so on) : 

1. 0-$2999 

2. $3000-4999 

3. $5000-6999 

4. $7000-8999 

5. $9000 or more 

4. Please circle yes or no if you have the following things in 
your home. 



1. 


Yes 


No 


Television 


2. 


Yes 


No 


Radio 


3. 


Yes 


No 


Daily Newspaper 


A. 


Yes 


No 


Magazines 


5. 


Yes 


No 


Dictionary 


6. 


Yes 


No 


Encyclopedia 


7. 


Yes 


No 


Children's Books 



5* How much schooling have you completed? 

1. Some grade school (less than 8 years) 

2. Completed grade school (8 years) 

3. Some high school (less than 4 years) 

4. Completed high school (4 years) 

5. Some vocational of business school after high school 

6. Completed vocational or business school after high school 

7. Some college 

8. Completed college 

9. Graduate or professional school 
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Select from the list below the category which best describes 
the job of the family bread winner. 

1. Unemployed 

2. Welfare recipient and uiAemployed 

3. Executives and proprietors of large concerns, major pro- 
fessionals (examples: doctor, lawyer, dentist, department 
store manager) • 

4. Managers and proprietors of middle size businesses and 
service professionals (examples: policeman, fireman, branch 
manager of grocery chain). 

5. Administrative personnel of large concerns, owners of small 
businesses (examples: clothing shop owner, IBM programmer, 
florist) . 

6. Clerical, retail sales workers, and technicians (examples: 
shipping clerk, dry cleaning shop attendant, x-ray techni- 
cian). • 

7. Skilled workers (examples: hair stylist, appliance repairman, 
electrician) . 

8. Semi-skilled worker (examples: gas station attendant, delivery 
man, domestic worker). 

9. Unskilled workers (examples: cafeteria or laundry work, car 
washer, farm hand). 

10. If none of the above fits the description of his or her job, 
please describe the job in the space provided below. 



What is your race/ethnic background? 

1. Black 

2. ^ White, Spanish last name 

3. White 

4. Oriental 

3. American Indian 
6. Other 
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INCENTIVES "IN EPUCATION 
STUDENT INTERVIEW 

Interviewer 



Date^ 

L ' J 



1. (a) Do you enjoy school? Tell me how you feel 

in the morning before coming to school — glad 
or not so glad, 

P FP NO FN N 

I I i_i I I 

(b) What do you like or dislike about your school? 

r n 

Record 



2. (a) When you get to school do you like the work 

you do in class? 

P FP NO FN N 

I I I I L_J 

(b) What is it you like or dislike about the work 
you do in school? 

Record 

u _| 

3. (a) How do you feel about doing work in arithmetic, 

do you enjoy it or would you rather not do it? 

P FP NO FN N 

I I I I L_J 

(b) What is it you like or dislike about arithmetic? 
Record 
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4* (a) Do you enjoy work in reading, or would you 
rather not do it? 



P FP NO FN N 

I I I I I I 

(b) What is it you like or dislike about reading? 
Record 



J 



5* (a) How many times do you visit the library on your 
o%m, out of class, once a %«eek, t%iice a week, or 
more? 

P FP OC R N 

I I I i » i 

4 3 2 1 0 
(b) What is it you like/dislike about your school 
library? 



1 

M 

Record 

L- -J 

6. (a) When you do your school work do you get any help 

!frpm your friends, if so, how many. times last %#eek 
4id you get help? 

. P PF OC R N 

I I I i i I 

4 3 2 1 0 
(b) Would you/do you like working with other students, 
;or would you/do you prefer to work on your own? 

f" -I 

Record 



-J 
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7. (a) Do you help your friends with their school 

viork, if so, how many times last week did 
you give help? 

F FF OC R N 

i_i I l_! I 

4 3 2 1 0 
(b) Would you/'do you enjoy being able to help 
other students? 

r" . • n 

Record 

L -J 

8. (a) Wbuld you like to be good at doing arithmetic 

problems or don't you care? 

P FP NO FN N 

I L^J I I I 

(b) What is it you like/dislike about arithmetic 
problems? 

r n 

Record 

L. J 

' 9. (a) Would you like to be a good reader? 

P FP NO FN N 

L I I I I I 

(b) What is it you like/dislike about reading? 

r 1 

Record 

i_ -J 
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10. (a) When you finish school do you expect to get 
a good job? 

P . FP NO FN N 

I I I I I I 

(b) What type of job would you like to have? 
Record 



J 



11 • (a) Does your teacher give you a hard time if 
your wrk is not good? 

F FF OC R M 

f4Qst Sgme Once 
«u ^tiiae txme^. txme vhile 
(b) What sort of thxngs aoes your teacher say or 

do if yotir work is not good? 

r n 

Record 



J 



12 • (a) Do your parents, your non or dad, give you 
a hard tine if your work is not good? 



F FF OC R M 

TI Host 'some ^ Onqi ■ ^ 
_ime time time while 
(b) What sort of things do your mom and dad say 

or do if your work is not good? 

r ti 

Record 



u 
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13. (a) Do you expect to be a good reader when you 
leave school? 

P FP NO PN N 

I I I 1 i i 

(b) What type of things do you v:ant to be able to 
read, e.g. newspapers, magazines, books? 

- Record 

i 

i 

14. (a) Do you expect to be good at arithmetic when 

you leave school? 

P FP NO PN N 

I I L_J I I 

(b) What sort of things do you expect to be able 
to do '.xth your arithmetic talent, e.g. say 
add up a shopping list, keep a check book, work 
in a store, teach, etc? 



' Record 



L- _l 



y 15. (a) How do you get along with your teacher, well, 
: or not so well, e.g. is she easy to talk to, 

;: can you ask her questions? 

t 

* P FP NO FN N 

I 1 I I \ I 



(b) What is it you like/dislike about your teacher? 
Record • 



u 



J 
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INCENTIVES IN EDUCATION 

TEACHER INTERVIEW 
i ^ i i ir City 



I f I I jBlda, 
LJJLLJTchr. 



I-I i f I Interviewer 

i — LJLLJinterview No. 
Date 



(a) Do you think the payment of incentives to 
teachers is a good thing? 

, P FP NO FN N 

I — I — I I I I 



(b) What specifically do you like/dislike about 
the idea of monetary incentives to teachers? 



Record 



2. (a) Do you think that the payment of incentives 
to parents is a good thing? 

. P FP NO FN N 
I 1 1 I l_J 

(b) What specifically do you like/dislike about 
the Idea of monetary incentives to parents? 



r 



Record 



1 



J 



3. We would be pleased, to receive your suggestions as to 
how this incentive project might have been improved. 
If you had been consiilted in the early stages of the 
project what changes would you have made? 



r 



Record 



ERIC 



J 
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4* (a) Have you made use of any techniques to in- 
dividualize the curriculum you use in the 
classroom if so, how often in a normal school 
%ieek would you use such techniques? 
F FF CX: R N . 

I I I l_] I 

4 3 2 1 0 
(b) Can you give me some exan^les of the sort of 
thing you have done? 

r n 

Record 



5, (a) How often in the last 2 months have you requested special 
materials (films, books, etc.), not normally 
supplied by the school district, to help you 
with your lessons? 

F FF CX: R N 

* I I i i 



4 3 2 1 0 
(b) How successful were you/are you in getting such 
materials, and what were/are the major problems, 
if any? 

r n 

Record 



6. (a) Are you satisfied with the profession of 
teaching for you as an individual? 

J? FP NO FN N . 

I I I I I I 

(b) What aye the sr^.clfic likes/dislikes you have 
about your joc jls a teacher? 

r n 

Record 



J 
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^ 7. (a) In the last 2 weeks have you ever considered trans- 
t ferring out of the school you are presently teaching 

I in? 

I F FF OC R N 

i I i I l_J 

4 3 2 1 0 
(b) What do you find particularly rewarding/un- 
rewarding about teaching in this school? 

r 1 

Record 

L_ -J 

8» (a) In a normal school week do you use peer tutoring as 

a classroom teaching technique, if so, to what degree? 

F FF OC R N 

I I l_l \ I 

4 3 2 1 0 
(b) Do you think that peer tutoring as a teaching 
technique could be usefully employed by the 
teacher? 

r "1 

Record 

9* (a) Do you enjoy yoxir students as children? 



I P FP NO FN N 

1 I I \ I [ I 



(b) What do you particulau:ly like or dislike 
about them? 



1 

I Record 
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10. (a) What % of your students' parents have visited you 
in the last two months, and for what reasons? 



FP OC 
J- 



N 

J I 



16+ 15-11 10-6 5-1 0 

(b) Do you encourage parents in any way to visit with 
you and discuss their child's progress? 



Record 



r 



L 



J 



11. 



(a) Do you feel you get on well with the parents 
of the children you teach? 



P FP NO FN N 

I I I i i i 

(b) What do you specifically like/dislike about 
the parents of the children you teach, and 
c^in you give any brief exaittples? 



Record 



r 



L 



J 



12. (a) Many politicians and community members claim 

that the most important thing in the classroom 
should be discipline. Do you agree? 

P FP NO FN N 
I 1 1 I t I 

(b) Do you find yourself faced with many discipline 
problems? Can you give some examples? 



Record 



J 
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13. (a) How do you feel about the ability of the 

parents of your pupils to help their children 
with their homework? 



Record 



P FP NO FN N 
1 III' 



I 



(b) Have you any experience of parents helping 

their children with homework, and how successful 
was such aid? 



r 



L. 



J 



14. (a) How wany hours, during a normal week, if it is 

necessary, do you spend on lesson preparation out of 
school and which subject areas take most time? 



Record 



L 



-L 



11+ 10-7 6-4 3-1 0 



(b) Do you find it necessary to spend much time on 
lesson preparation out of school time? 



r 



1 



J 



15. In teaching reading to your students do you attempt 
to concentrate on any particular type of student, 
for e.g. the ver;^ bright, the very poor or those 
students in between? Can you elaborate on any 
strategies of this type that guide you in your 
teaching? 



Record 



L 
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Record 



•6- 



16, In teaching arithmetic to your students do you 
attempt^ to concentrate on any particular type of 
student, for e.g. the very bright, the very poor, 
of those students in between? Can you elaborate 
on any strategies of this type that guide you in 
your teaching? 



r 



17, How much use does your class make of the school and 
public libraries? Do you find it necessary to offer 
your students encouragement in order that they use 
the library; if so, what form does it take? 



r 



~1 



Record 



I8. (a) What are your feelings towards the incentive 
project being tried in your school district? 

P FP NO FN N 

I I I I I ( 

(b) What in particular do you like/dislike about 
the project? 



r 



Record 



L. 



19 • Do you have any suggostions for any other forms 
of incentives that r.ight be used to encourage 
academic achievement gains in students? 



r 



Record 



ERIC 



U 
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INCENTIVES IN EDUCATION 



* PARENT INTERVIEW 

Parent Kamfe 



Interviewer 

-J 

Relation to child_ Date 

(a) Are you happy with the school yotir child is 
attending this year? 

(b) What is it you like/dislike, are happy/unhappy 
with about the school? 

P FP NO FN N 

I I I I I I 

(a) Do you attend school activities, e.g. PTA meet^ 
ings, or other school activities? . 

(b) How many times have you visited the school 
in the last last two months? 

F FF OC R N . 

I I I J ! I 

4 3 2 1 0 



(a) How often do you encourage ypiir child to do well 
at' his school' work? 

(b) How many times during the past week have you offered a 
form of encouragement? 



(a) Do you help your child in doing his/her 
homework? 

(b) How many times last week did you help your 
child in his/her homework? 

F FF OC . R N 

I I I I I 1 

4^ 3 2 0 

(a) Do you think that groups of parents working 
together can improve their children's school 
grades? 

(b) Have you had any experiences that make you feel 
that way? 

P FP NO FN N 

I \ 1 I I I 
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6« (a) Haye you organized any extra-help sessions 

for reading/arithmetic work for your children, 
with other parents in the neighborhood? 

* (b) How often in the last month have such sessions been 
held? 

4 3 2 1 0 
(c) Who has taken' the lead, to your knowledge, in 
organizing parent groups? Can you give roe any 
names of parents you know? 



Record 



7. (a) Are you satisfied with how your child is doi ng 
in arithmetic? 

(b) What do you feel is good/bad about what is 
being taught? 



8, (a) Are you satisfied with how your child is doing 

in reading? 

(b) What do you feel is good/bad. about what is 
being tavight? 

, P , rp NO FN N , 

1 \ [_J !_| 

9. (a) Do you get extra reading and/or arithmetic 

learning materials for your child? 

* (b) How many materials have you obtained in the last 
2 months? 

F , PF OC R N , 

I \ i i i I 

3 2 1 Yes but 0 
not in 
the last 
2. months 
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10; (a) Do you encourage your child to use the 
public library? 

* (b) How often have you encouraged your child in 
this way in the last two weeks? 

. F FF . OC . R . N 



4 3 2 1 0 

11. (a) How often does your child use the public/ 

school library? 

* (b) How often has he brought books home in t|ie last 

two weeks? 

4 3 2 1 0 

12. (a) Do you think parents need to be familiar 

with their child's teacher? 

(b) Dp you feel you know your child's teacher as well as 
you would like? 

I P FP , NO FN N 

I I \ l—l I 

13* (a) Have you visited with or met your child*s 
teacher? 

* (b) How many times in the last two months have 

you met or talked with your child's teacher? 

, F FP DC R N 

I I \ L_l I 

4 3 2 1 0 

14* (a) Do your child's friends work with your child 
on their reading/arithmetic homework? 

* (b) How often has that occurred in the las^t two weeks? 




15* (a) Do any adults outside of your home help your 
child with his/her homework? 

* (b) How often in the last two months has such help 
been given? 




-4- 



16. (a) Do you approve of the way the school is 

teaching your child to behave? 

(b) What particularly do you like/dislike? 
. P , PP -NO > FN ; N 

I I' III I 

17. (a) Do you approve the idea of payment of money 

being paid to teachers, e.g. a bonus, if your 
child learns more? 

(b) What do you particularly like/dislike? 




18. (a) Do you like the idea of payments of noney being 

paid to parents if their child learns more due 
to parental help? 

(b) What psurticularly do you like/dislike aix>ut 
i:he Idea? 

. P PP NO FN . N 

I I 11 I I 

19. (a) Have .you discussed or been to meetings with 6j:her 

parents about the payments of money project? 

* (b) H6w many times have you had such discissions/ 
meetings in the last two weeks? 




20. (a) Have you discussed or attended meetings about 

the payments of money project with your child's 
teacher or other school official? 

* (b) H6w many times have you had such discussions/- 
meetings in the last two weeks? 

F FF OC R N 

i I I I I I 

4 3 2 1 0 



f- 
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21. If you could have had a say in setting this 

project up, how vibuld you have done it differently? 



r 



Record 



L. J 

22* iDo you have any suggestions for other forms of 

incentives that might be used to stimulate activity 
axid learning, in the classroom? 

r : 

Recoxxl 
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APPENDIX B 

TABLES OF ACHIEVEMENT DATA RESULTS 
GRADE BY GRADE 



TABLE B-1 
READ STAM GAIN 

/ 
/ 



GRADE 


IIEAN 1 


SIGMA 1 


HUM 1 


MEAN 2 


sicm 2 


NUM 2 






trVALUE 


R-SQUARED 


1 


8.3864 


4.1271 


44.0 


9.9219 


4.2810 


64.0 






-1.8583 




2 


8.3333 


4.7907 


. 48.0 


10.27.78 


5.3214 


54.0 






-V.92?? 


. .0359 


3 


5.6222 


6.0988 


45.0 


5.4262 


3.8878 


61.0 






+0.2017 




4 


5.4021 


5.1774 


97.0 


4.4063 


4.3597 


64.0 






+1.2698 


.0100 1 


5 


5.4231 


4.2197 


78.0 


3.0339 


6.5862 


59.0 






+2.5806 


.0470 


6 


3.7835 


5.5775 


97.0 


5.5167 


6.6243 


60.0 




_ 


-1.7595 


.0196 - . 



+ 



JACKSONVILLE 



GRADE 


MEAN 1 


SIGMA 1 


NUM 1 


MEAN 2 


SIGMA 2 


NUM 2 


t-VALUE 


R>SQUAREb 


1 


9.3750 


4.0739 


112.0 


10.3506 


4.3279 


. 77.0 


-1.5770 


.0131 


2 


6.5865 


5.2644 


133.0 


10*0886 


6. 1231 


79.0 


-4.4037 


.0845 


3 


7.7231 


5.4956 


130.0 


5.7742 


5.0025 


93.0 


+2.7096 


.0322 


4 


6.2710 


6.6851 


107.0 


4.9524 


4.8230 


126.0 


+1.7436 


.0130 , 


5 


4.4420 


5.9878 


138.0 


4.4797 


5.0702 


148.0 


-0.0576 


.0000 


6 


7.4492 


7.2861 


118.0 


1 6.1471 


5.5263 


170. 0 


+1.7233 


i0103 . . 



TABLE B^l (Cont'd.) 



sam Antonio 



GR/JJE 


i-SAN 1 


SiGI-l\lL KUM 1 






^EAN 2 


SIGMA 2 


IIUM. 2 1 1 




t-VALUE 


R-SQUARED 


1 


10.5065 


5.4619 


77.6 






14.7273 


6.9829 


44.'0^ 








-3.6881 


.1026- 


2 


10.0625 


5.9830 


16.0 






8.7045 


4.8780 


44.0 








+0.8969 


.0137 


3 


7.3944 


5.4941 


71.0 






6.3607 


- - 

4.0948 


61.6 








+1.2088 


■ .0111 


4 


7.3571 : 


7.7938 


56.6 






6.1154 


6.4585 


52. o; 








+0.89''7 


.0075 


5 


5.3542 


9.2977; 


.48.0 1 




5.4222 


7.-2252 


45.0 








-6.0392 


.0003 


6 


' 8.0615 


7.6053 


65.0: 






5.2857 


5.4740 


. 42.6 








+2.0462 


: .0383 



OAKLAND 



GR^'DE 


i MEAN 1 


SIGMAlt NUM I 1 


|: MEAN 2 : 


SIGMA 2 


KUM 2 




t-VALUE 


R-SQUARED 


1 


10.2381 


5.8446; 


63.6 


9.6175 


4.5845 


• 57.0 




+1.2636 


: .0134 


2 


;iO;737;7. 


6.1479 


61.0 


^ 5.6538 


6. 31 19 


■ 78.0 




^ +4.7662 


. 1422 


3 


7.1087 


'4.6678 


: 46.0 


4i2870 


5.3233 


108.6 




+3.1192 


.0602 


4 


4.6966 


:5.4050 


89.0 : 


5*8116 


5. 9663 


69.0 = 




-1.2289 


.0096: 


5 


3. 6863- 


4.6454 


51.0 


4.1250 


6.1979 


; 86.0 




-6.4336 


iOOlS 


6 


6.7843 


6.1198 


51.0 


: 3.2105 


5.0632 


95.0 




+3.7752 


.0901 ; 
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TABLE B-2. 
MATH STAN -GAIN 





' ••' -- . ■ . ■ _ • 

- f J 
CINCINMTT 




GRADE 


MEAN: I 


SIGM\ 1 


KUM i 




JEAN 2 ; 


SIGi-I/V 2 


NUJI 2 






t-VALUE 


R~SQUARED 


1 


15.8636 


7.6206 


44.0 




15.1538 


8.380i 


65.0 








.007 0 


2 


15.8980 


8.1400. 


49.0 


15.0800 


7.0994 


50.0 


+0.5332 




3 


5.5iir: 


9.0745 


45.0 




9.1803 


6.2544^ 


61.6 


-2.4644 


.,...0552 .. 


4 


■ 8.5567 


6.6536 


- 97 iO 




9.3810 


6.6099 


63. 6 






-6. 7676 


' .0037 , 


5 


6.9359 


5.9157 


78.0 




'4,8644 


6.3990 1 .59.0 


>1;9592 


i6276 


6 


3. 8247 = 


4 i 9456 


- 97.0 




1 4.1 5^0 ; 


5.8363 - 


66ib - 


-0.3735 


.0009-. 





■ — - - JACKSONVIIiLi: . 


GRADE 


MEAN r 


SIGMA 1 


KUM 1 




= MEAN 2 


SiGI-lA 2 


KUM 2 




1 t-VALUE 


R-SQUARED 


1 


15.8319 


8.1579 


113.0 


16.4416 


6.8739 


77.0 


-0.5383 


.0015 


2 ' 


11.4885 


8.4747 


131.0 




13.3590 


7.5918 


78.0 


: -1.6032 


.0123 


I \ 


: 9.8682 


7.6110 


. 129.0 


7.8901 


8.0090 


91.0 


: +1.8578 


.0156 


4 


: 9.3619 


6.1082 


; 105.0 


1 

I 


8.1453 


6.5182 


117.0 


+1.4303 


.0092 


5 


6.5374 


5.3098 


138.0 


6.5374 


5.7616 


147.0 




-0.4206 


.0006 : 


6 


7.2288 


5. 7041 : 


, 118.0 


4.4024 


4.2836 


; 169.0 


: +4.7920 


.0746. 



ERJC 



B-4 



TABLE B^2 (CohtVd.) 



OAKLAND 



GRADE 


, MHAN 1 : 


SIGM^it NUM 1 


1 ^SAN 2 


I siGMV 2 


NUJ-1 2 


1 ;t~VALUE 


R-SQUARED 


1 


15.9048 


7. 8734 


63.0 


I II.929J 8.026^ 


- 57.0 


4-2.7364 


; ,05?Z. 


2 


14.4167 


6.4500. 


:60.0.. 


11.435r 


8 . 966c 


■ 78.0 • 


+2. 188S 


.0340 


3 


: 10.2889, 


6.3160 


i 

45.0 


6.694/^ 


6.915: 


108.0^^ 


+3.002? 


.0564-^ 


4 


Z.0341 


6.9024 


88.0 


6.285/! 7.3171 


'70.0 


+0.6592 


- .0028 


5 ^ 


■ 4.8824' 


- 5*4430. 


5i.o; 


3.,400C 


6.4052 


80. 0_ 


- +1.3673 


.0U3 


6 


7; 0600 


5.4637 


50.0 


2.9052 


5.1075 


95.0 1 


' +4.5634 


.1271 







. SAN- ANTON TO 


GRADE 


msAn 1 


SIGm 


jlNUMI 


; ^EAN 2- 


SiGKA 2 


I'JUM 2 


1 t -VALUE 


R-SQUA_KEb 


1 


22.3636 


Li .7 920 


77.0 : 


_ t7v6455 


1.1X11 


44.6 ; 


= +2.6795 


0569 : 


2 


11.8125 


6.6797 


16.0 


12.3182 


7.6759 


■ 44. o| 


-0.237^ 


.0010 


3^ ; 


= 9; 1831 


6 i 5956 


' 71.0 : 


6,4262 


8.0093 


61. oi 


! ■ +2.168C 


.0349 


4 


13.4464^ 


: 6.7012 


: . 56.6 


' 5.:6538 


7.0984, 


52.0! 


1 +5. 8684 


.2452 : 


5 ; 


Ji0816 


7.1439 


49.0 


1 6.4773 


4.9860 


44.0 


1 +1.2324 


.0167 


6 


8.2615" 


7.4084 


65.6i 


j 6.5476" 


4.4018 


42.0 


' +1.351/ 


- .0171 



TABLE B-3 
READ STAN PRE AND POST 
ClNCiNNATI PRE 



+ 



CINCINNATI 



GRADE 


MEAN 1 


SIGMA 1 


NUM 1 




MEAN 2 ; 


SIGMA 2 


injM 2 




t-VALUE 


R-SQUARED 


1 


26.2727 


4.5257 


44. 0 




24.6406 


3.5746 


64.6 




+2.0900 


.0396 


2 


34.1250 


6.3300 


48. 0 




: 35. 5370 


8.3750 


54.0^ 




-0.9511 


.6696 


3 


47.2667 


10.2523 


; 45. 0 




47.9836 


' 7.3813 


61.0 




-6.4188 


.6017 


4 


50.4639 


9.2512 


97.6 




53.8281 


9.0423 


64.0 




-2.2784 


.63i6 , 


5 


55.7564 


li.7939 


78.0 




60.2373 


8.4228 


59.0 ; 




-2.4782 


.0435 


6 


63.2165 


12.5874 


; 97.0 




60.9833 


112.0613 


60.0 




+1.0974 


.0077. 



CINCINNATI POST 



GRADE 


MEAN 1 


SIGtiA 1 


NUM 1 




MEAN 2^ ; 


SIGMA 2 


NUM 2 






t-VALUE 


R-SQUARED 


1 


34.6591- 


6.1830 


^ 44.0 


34.5625 


5.4274 ^ 


64.0' 






+0.0858 


.0061 


2 


42.4583 


9.0294 


48.0 




45.8148 


6.9908 


54.0 






-2.1114 


.0427 


3 


52.8889 


-9.8170 


45.0 




53.4098 


7.4931 


61.0 






-0.3099 


' .0009 


4 


55.8660 


.9.8156 


97.0 




58.2344 


9.4948 


64.0^ 






-1.5178 


.0143 


5 


i 1.1795 


12. 7266 


78.0 




63 i 27 12 


10.9463 


59.0 






-1.0107 


.0075 


6 


67.OOO0 


13.8346 


97.0 




66.5006 


11.3055 


60.0 






+0.2354 


.0004 



TABLE B'-^3 (Cont'd.) 
JACKSONVILLE PRE 



JACKSONVILLE 



GRADE 


mm 1 


SIGMA 1 


NUM 1 


MEAN 2 


SIGm 2 


_NUM 2 ; 


t-VALUE 


R-SQUARED 


i 


22:3929 




U2.Q 




5.1564 


77.0 


+0.7115 


.0027 


2 


28.7669 


6.1483 


- 133.0 


32.0127 


6.3841 


79.0 


-3.6637 


.0601 


3 


35.3615 


7.7660 


130.0 


41.8065 


9.6169 


93.0 


■^5.5276 


! .1215 


4 


42.8879 


12.1207 


107.0 


: 48.3651 


9,7444 = 


126.0 


-3.8226 


.0595, 


5 - / i 


51.6014 


11.4398 


138.0 


r52 . 7432 


10.0895 


148; 0, 


. -0.8966 


.0028 


6 


54.6356 


14.4835 


: 118.0 


; '59. 3647 


12.0829 


170.0 


-3.0087 


L .0307:. 


"1 




-JACKSONVILLE POST 






JACKSOWTLLK 


GRADE 


mm I 


SIGM 1 


: NUM 1 ; 


MEAN 2 ; 


SIGMA 2 


: NUM 2 


t-VALUE 


R-SQUARED 


i 


^31 . 7679 - 


-4.3949 ' 


112.0 


32.2208 


?.8075 


77. Q 




.nn^n 


2 


-35.3534 


'8.'3713 


133.0; 


^ 42 i 1013 


9.9854 


- 79.0 


-5 i 2756 


.1170 


3 


43.0846 


8. 9443: 


130.0 


=.47.5866 


10.7383 


93.0 


-3.4018 


.0498 


A 


49.1589 


L2.7162 


107. 0 


^53.3175 


10.5061 


126.0 


-2j7341 


.0313, 


5 


56.0435 


L3.4T47 


138.0 


57.2230 


11.6203 


. 148.0 


-0.7962 


.0022 


6 


62.0847 


15.5517 


118.0 


65.5118 


14.0953 


170.0 


-1.9445 


.0130. 



B-7 



TABLE B-3 (Cont'd.) 

OAKLAND PRE 



OAKUND 



GRADE 


MEAN 1 


SlCmij: NUM 1 ,| 1 1 lEAHAZ: 


SIGMA 2 


NUM 2 


1 1 U-VALUE 


R-SQUARED 


1 


26.0000 


4.8957 


63. o| 


27.5088 


4.1193 


57. o! 


-1.816 


» .0272 


2 


36.4754 


8.6170 


61. o| 


' 36.2051 


8.4306 


78. c : 


+0.1858 


.0003 


3 


49.4783 


8.3898 


46. d 


49.9537 


9.4150 


108.0- 


-0.2960 


.0006 


4 ■ 


51.3034 


12.4082 


89. a 


56.1739 


9.4011 




-2.7118 


.0450 


5 


58.-1765 


10.7419 


51. ol 


60.9750 


12. 1749 


80. a 


-1.3417 






'"'68.7451 


15.2156 


51. 


68.5789 


10.4256 


95. oil 


+0.0778 


.0000 


OAKLAND POST 






OAKLAND 


GRADE 


MEAN I 


SIGMAl 


NUM 1 1 


jiEAN 2- 


SIGm 2 


; NUM 2 ; 


^ t-VALUE 


R-SQUARED 


1 


36.2381 


:5'.7043: 


63.0/; 


36. 5263 


4.6679 


57.0 ' 


-0.3010 


.0008 


2 


47.2131 


8.1017 


61.0 


41.8590 


7.1926 


. ■ 78.0 


+4.1195 


.1102 


3 


56.5876: 


.8.1937 


46.0: 


54.2407 


8.9588 


108.0 


+1.5248 


.0151 


4 


"56^00005 


^ 12. 7618 


89.0^ 


61.9855 


12.8252 


69.0 


= -2. 9177 


.0517 


5 


61.8627 


12.1557 


51.0 


65.10bC 


10.9077 


80.0 


r -1.5837 


.0191 


6 


75.5294 


13.9561 


51.0 ; 


71.7895 


11. 1354 


95.0 


= +1.7675 


.0212 



B-8 



6 

ERIC 



TABLE B-3 (Cont'd.) 
' SAN ANTONIO PRE 





. . SAN ANTC .^u 


GRADE 


MEAN 1 


SIGMAl 


: NUM 1 


MEAN 2 : 


SIGMA 2 


KUM 2 1 


i t-VALUE 


R-SQUARED 


1 


24.8701 


4.1082 


77.0 


22.5000 


A. 6929 


44.0 


' ' - 

+2.8974 


.0659 


2 


27.6875 


5.1990 


16.0 


32.4773 


-5.1466 


44.0 


-3.1795 


.1484 


3 


A3. 225A 


10.9299 


71. 0' 


; 45.2623 


9.7637 


61.0 


-1.1210 


.0096 


4 


49.8036 


9.5678 


56.0 


. 52.730f 


11.3865 


52.0 


-1.4500 


.0195 


5 


55.3542 


13.2705 


48.0 


:62.800C 


B.7024 


45.0 


-2.6618 


.0722 


6 


59.8769 


12.3876 


65.0 


^70.619C 


9.8227 


42.0 


-4.7369 


.1761 



SAN ANTONIO POST 







SAN ANTONIO 


GRADE 


^5EAN 1 


SIGMAl 


i NUM 1 


1 MEAN 2 


SIGMA 2 


NUM 2 1 


t-VALUE 


R-SQUARED 


1 


35.3766 


5.4048 


77.0 


37.2273 


5.7258 


44.0^ 


-1.7731 


.0257 


2 


.37.7506 


6.6383 


16.0 


4l.:18l£ 


7.5491 


• 44. 0 


-1.6049 


.0425 


3 


50i6l?7 


12.0823 


7i.O 


51.623C 


9.4836; 


61.0 


-0.5243 


.0021 


4 


57.1607 


8.9ii2 


56.0 


58.8462 


12.9923 


52.0 


-0. 7910 


.0059 


S 


60.7083 


13.7465 


48.0 


68.2222 


14.3272 


45.0 


-2.5809 


• .0682 


6 


67.9385 


13.6426 


? 65.0 1 


75.9048 


11^2549 


42.0 


-3 .1526 


.0865 



t 

! 

o ! 
ERIC 



TABLE B-4 
MATH STAN PRE AND POST 
CINCINNATI PRE 



GRADE 


I - - - 
MEAN 1 


SIGMA I 


NUM 1 


MEAN 2 


SIGVA 2 


NUM 2 


t-VALUE 


R-SQUARED 


1 


23.1591 


6.3207 


44.0 


22.1077 


5.2154 


65.0 


+0.9473 


.0083 


2 


33.8367 


5.3942 


49.0 


36.1400 


9.7248 


50.0 


-1.4532 


i0213 


3 


46.5556 


13.5740 


45.0 


, 51.1311 


10.4920 


61.0 


-1.9577 


.0355 


4 


55.6495 


12.1793 


97.0 


57.7143 


10.2979 


: 63.0 


-raii8 


.0078, 


5 


66.1667 


11.3531 


78.0 


68.4746 


8.0502 


59.0 


-1.3285 


.6129' 


6 


72.2784 


11.2997 


97.0 


76.8500 


10i3003 


60.0 


;i-2.5466 


.0402 



ClfiCIiTtiAII POST 



CINCINNATI 



GRADE 


13EAN t 


SIGIIA 1 


NUM 1 


MEAN 2 


SIGMA 2 


NUM 2 




t-VALUE 


R-SQUARED 


1 


39.0227 = 


11.6349 


44.0 


37.2615 


11.6113 


65.0 




+0.7763 


.0056 


2 


49.7347 


10.4458 


49.0 


51.2200 


11. 7635 


50. 0 




-6.6658 


.0045 


3 


52.0667 


12.2147 


45.0 


60.3115 


11.2480 


61.0 




-3.5962 


.1106 


4 


64.2062 


13.3432 


97.0 


: 67.0952 


13.1722 


63.0 




-1.3448 


.0113\ 


5 


73.1026 


11.8086 


78. 0 


73.3390 


8.9532 


59. 0 




-0. 1284 


.0001 


6 


76.1031 


11.9412 


97.0 


_ 1 

81.0000 12.5333 


60.0 




-2.4499 


_ .0373: 



B-ib 



TABLE B-4 (Cont'd.) 
JACKSONVILLE PRE- 



+ 



JACKSONVILLE 



GRADE 


MEAN 1 


SIGliA 1 


NUM 1 




MEAN 2 


SIGMA 2 


NUM 2 






t-VALUE 


R-SQUARED 


1 


18.2124 


5.4714 


113.0 




17.0000 


6.3702 


77.0 






+1.4021 


.0103 


2 


29.3511 


7.823ff 


131.6 




29.8846 


7.8773 


78.0 






-0.4756 


.0011 


3 


36 . 7442 


9.1244 


129.0 




43.4615 


9.9580 


91.0 






-5.1774 


.1095 


4 


48.5905 


12.0517 


105.0 




51.8462 


11^4339 


117.0 






-2.0647 


.0190 , 


5 


58.7681 


12.4633 


138.0 




61.7959 


12.7553 


147.0 






^2i0250 


.0143 


6 


69. 1271 


14.3268 


118.0 




71.3787 


12.5389 


169.0 






-1.4110 


.0069. 



JACKSONVILLE POST 
JACKSONVILLE 



GRADE 


; MEAN I 


SIGH\ 1 


NUM 1 




^IEAN 2 


SlGMk 2 


NUM 2 




t-VALUE 


R-SQUARED 


1 


34,0442 


'{8.9297 


113.0 


33. 44 If 


10.143C 


77.0 




+0.4321 


.0010 


2 


40.8397 


12.2593 


131.0 




43.243.C 


11.2852 


78.0 




-1.4117 


.0095 


3 


46.6124 


11.2592 


129.0 




51.351C 


13.1727 


91.0 




-2.8644 


.0363 


4 


57.9524 


13.9404 


^. :0 




59.9915 


12.1673 


117.0^ 




-1.1636 


.0061, 


5 


65.0290 


13.4858 


138.0 : 




68.^333 


13.0905 


147.0. 




-2.0987 


.0153 


6 


76.3559 


15.6977 


118.0 




75.7811 


13.4790 


169.0 




+0.3321 


.0004 



B-ii 



TABLE B-4 (Cont'd.) 
OAKLAND PRE 



OAKT>AND 



GRADE 


MEAN 1 


SIGMAl 


:. NUM 1 


1 MEAN 2 


SIGMA 2 


NUM 2 1 


,|t-VALUE 


R-SQUi\RED 


1 


21.7937 


4.9192? 


63.0 


25.2632 


4.6657 


57.0 


-3.9536 


.1170 


2 


36.6667 


9. 1719 


60.0 


35.871E 


9.1982 


■ 78.0 


+0.5039 


.0019 


3 


51.8889 


8.5579 


45.0 


: 50.8981 


11.352^ 


108.0 


+0.5261 


.0018 


4 


54.9318: 


14.4991 


88.0 


60.200C 


12.3d6C 


70.0 


-2.4235 


.0363 


5 


62.1961 


12.127i 51.0 


70.625C 


13.5173 


80.0 


-3.6195 


. .0922 


6 


78.7800; 


13.331^ 50.0 


77.5895 


12.0711 


95.0 


+0.5444 


.0021 



OAKLAND POST 







OAKLAND 


GRADE 


MEAN 1 


SIGMAl 


- NUM I 


MEAN 2 


SIGMA 2 


NUM 2 II 


i|t-VALUE 


R-SQUARED 


1 


37.6984 


9.0708 


63.0 


37. 1930 


9i2185 


57.0 


+0.3025 


.0008 


2 


51.0833 


11.1085 


60.0: 


47^3077 


10.3426 


78.0 


+2.0584 


.0302 


3 


62.17.78 


9.4515 


45.0 


57.5926 


11.9203 


108.0 - 


+2.2957 


. .0337 


4 


61.9659 


12.7950 


88.0 


66.4857 


13.4862 


70.0 


-2.1535 


.0289 


5 


67.0784 


13.2843 


51.0 1 


74.0250 


12.3555 


80.0 


-3.0469 


.0671 


6 


85.8400 


12.1594 


50.0 1 


80.4947 


11.0134 


95.0 


+2.6792 


.0478 



B-12 



TABLE B^4 (Cont'd.) 
SAN ANTONIO PRE 



SAN ANTONIO 



GRADE 


MEAN 1 


SIGMal 


:. NiiM I 


]meAN 2 


SIGMA 2 


NUiM 2 






t-VALUE 


R-SQUARED 


1 


22.8831 


6.7609 


77.6 


21.7955 


6.1553 


44.0 






+0.8789 


.0064 


2 


31.3125 


4.8404; 


16.0 


35.4773 


6.4861 


44.0 






-2.3375 


-0861 


3 


47.8310 


12.5732 


71.0 


50.5410 


11.0839 


61.0 






-1.3035 


.0129 


4 


50.7857 


11.7206 


56.0 


58.3269 


12.3220 


52.0 






-3.25,??, 


.09U 


5 


65.6939 


11.8677 


49.0 


72.7727 


11.7748 


44.0 






-2.8826 


.0S37 


6 


71.3538 


14.3390 


65.0 


78.5476 


9.8233 


42.0 






-2.8461 


.9716 



SAN= ANTONIO POST 



SAN ANTONIO 



GRADE 


MEAN 1 


SIGMU 


:. NUM I 


MEAN 2 


SIGbA 2 


KUM 2 


t-VALUE 


R-SQUARED 


1 


45.2468 


11.324^ 


77.0 


38.8409 


7.4707 


44.0 


+3.3551 


.0864 


2 


43.1250 


6.8593 


16.0 


47.7955 


" 9.770i 


" 44.0 - 


-1.7566 


.0505 


3 . 


57.0141 


14.0667- 


71.0 


56.9672 


11.9345 


61.0 


+6.0205 


iOOOO 


4 


64.2321 


9.2402 


56.0 


63.980E 


13.4725 


52.0 


+0.1138 


.0001 


5 


73.7755 


11.7052- 


49.0 


79.250C 


13.345^ 


44.0 


-2.1675 


.0465 


6 


79.6154 


13.8932 


65.0 


85.0952 


8.5762 


42.0 


-2.2879I .0475 



B-13 



